From d96057196a397c7226754cca497046f155a546fa Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 30 Sep 2020 09:48:57 +0900 Subject: [PATCH 001/121] Starting incremental index support work This work will be moved forward on `bringup-25` branch for a while. And teh branch will be merged to master in the future. --- contrib/fssim/README.md | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 contrib/fssim/README.md diff --git a/contrib/fssim/README.md b/contrib/fssim/README.md new file mode 100644 index 00000000..6a6ac332 --- /dev/null +++ b/contrib/fssim/README.md @@ -0,0 +1,3 @@ +## This is the PoC code for incremental index support + +Incremental index is the one of the biggest item for LTFS Format Spec 2.5. From ac46fb252625965df0c521947ea5ec70cce006a1 Mon Sep 17 00:00:00 2001 From: DavidAPease Date: Thu, 1 Oct 2020 13:29:10 -0700 Subject: [PATCH 002/121] Initial checking of v1.5 fssim code --- contrib/fssim/README.md | 9 +- contrib/fssim/docs/README.txt | 391 +++++++++++++ contrib/fssim/src/fscommon.py | 713 ++++++++++++++++++++++++ contrib/fssim/src/fsglobals.py | 10 + contrib/fssim/src/fsidxupd | 159 ++++++ contrib/fssim/src/fsruntest | 74 +++ contrib/fssim/src/fssim | 376 +++++++++++++ contrib/fssim/testcases/daptest.txt | 11 + contrib/fssim/testcases/daptest2.txt | 21 + contrib/fssim/testcases/daptest3.txt | 10 + contrib/fssim/testcases/daptest4.txt | 7 + contrib/fssim/testcases/daptest5.txt | 34 ++ contrib/fssim/testcases/daptest6.txt | 9 + contrib/fssim/testcases/fssimin-Log.txt | 14 + contrib/fssim/testcases/fssimin-TI.txt | 77 +++ contrib/fssim/testcases/fssimin10.txt | 63 +++ contrib/fssim/testcases/fssimin4.txt | 7 + contrib/fssim/testcases/fssimin5.txt | 8 + contrib/fssim/testcases/fssimin6.txt | 8 + contrib/fssim/testcases/fssimin7.txt | 35 ++ contrib/fssim/testcases/fssimin8.txt | 18 + contrib/fssim/testcases/fssimin9.txt | 17 + contrib/fssim/testcases/logerr.txt | 19 + contrib/fssim/testcases/tc001.txt | 14 + contrib/fssim/testcases/tc002.txt | 8 + contrib/fssim/testcases/tc003.txt | 29 + contrib/fssim/testcases/tc004.txt | 34 ++ contrib/fssim/testcases/tc005.txt | 13 + 28 files changed, 2186 insertions(+), 2 deletions(-) create mode 100644 contrib/fssim/docs/README.txt create mode 100644 contrib/fssim/src/fscommon.py create mode 100644 contrib/fssim/src/fsglobals.py create mode 100755 contrib/fssim/src/fsidxupd create mode 100755 contrib/fssim/src/fsruntest create mode 100755 contrib/fssim/src/fssim create mode 100644 contrib/fssim/testcases/daptest.txt create mode 100644 contrib/fssim/testcases/daptest2.txt create mode 100644 contrib/fssim/testcases/daptest3.txt create mode 100644 contrib/fssim/testcases/daptest4.txt create mode 100644 contrib/fssim/testcases/daptest5.txt create mode 100644 contrib/fssim/testcases/daptest6.txt create mode 100644 contrib/fssim/testcases/fssimin-Log.txt create mode 100644 contrib/fssim/testcases/fssimin-TI.txt create mode 100644 contrib/fssim/testcases/fssimin10.txt create mode 100644 contrib/fssim/testcases/fssimin4.txt create mode 100644 contrib/fssim/testcases/fssimin5.txt create mode 100644 contrib/fssim/testcases/fssimin6.txt create mode 100644 contrib/fssim/testcases/fssimin7.txt create mode 100644 contrib/fssim/testcases/fssimin8.txt create mode 100644 contrib/fssim/testcases/fssimin9.txt create mode 100644 contrib/fssim/testcases/logerr.txt create mode 100644 contrib/fssim/testcases/tc001.txt create mode 100644 contrib/fssim/testcases/tc002.txt create mode 100644 contrib/fssim/testcases/tc003.txt create mode 100644 contrib/fssim/testcases/tc004.txt create mode 100644 contrib/fssim/testcases/tc005.txt diff --git a/contrib/fssim/README.md b/contrib/fssim/README.md index 6a6ac332..d9200886 100644 --- a/contrib/fssim/README.md +++ b/contrib/fssim/README.md @@ -1,3 +1,8 @@ -## This is the PoC code for incremental index support +# LTFS File System Simulator (fssim) + +The major change in version 2.5 of the [LTFS Format Specification](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) is the addition of Incremental Indexes. Incremental Indexes allow an LTFS index written to the Data Partition of the tape to contain only the changes from the prior index, rather than the full state of the file system. For a full discussion of Incremental Indexes, see Appendix H of the Format Specification reference above. + +When we first proposed Incremental Indexes there was considerable discussion concerning what they would need to contain, and how they would be implemented. I decided to write a simulator that would mimic file system operations, and create full and incremental XML indexes very similar to those in LTFS as a Proof of Concept. The simulator package also includes programs to merge such directories and verify the result, and a directory of test cases built by various members of the LTFS project. + +Full documentation of the programs is contained in the README file in the docs directory. -Incremental index is the one of the biggest item for LTFS Format Spec 2.5. diff --git a/contrib/fssim/docs/README.txt b/contrib/fssim/docs/README.txt new file mode 100644 index 00000000..90682776 --- /dev/null +++ b/contrib/fssim/docs/README.txt @@ -0,0 +1,391 @@ + +LTFS File System Simulator (fssim) Usage Guide and Examples (README) + +Overview +======== + +The File System Simulator programs are used to simulate file system activity, +create full and incrmental indexes reflecting that activity, update a full +index using incremental indexes, and verify the correctness of that update. + +The first program in the set is fssim. It simulates file system activity such +as creating or deleting files and directories, moving and copying files or +directories, listing information, etc. It also writes full and incremental +XML indexes to text files on request. + +The second program is fsidxupd (perhaps a poor choice of names). It updates +a full index created by fssim with one or more incremental indexes created +sequentially in the same execution of fssim, then creates a full index +from the result. The resulting full index should exactly match the +corresponding full index from fssim. + +The programs were written by David Pease at IBM's Almaden Research Center in May-August of 2018, and significantly revised and improved by the original author (now retired) in September of 2020. + +Invoking the programs: +====================== + +The programs are python executables, meaning that they can be invoked from +the command line in Unix-like environments (such as Linux and Mac) as long as +the executable bit is on and /usr/bin/python points to the python interpreter. +The programs are written so that they run correctly in either python 2 or +python 3. + +In Windows, it should be possible to invoke them by a command such as "python +fsXXX" or by renaming the two program files to fsXXX.py and invoking them by +name (if python is properly configured). + +There are two additional files besides the fssim and fsidxupd programs. They +are fscommon.py and fsglobals.py. Both of these are imported by the two +command programs; all four files should reside in the same directory. + +The format of the fssim command is: + + fssim [-d] [command file] + +where: + -d: prints a ton of debugging messages (probably meaningfull + only to me) during processing + command file: executes commands from the specified text file at startup + + A command file is simply a text file containing commands as they would be + entered from the keyboard; blank lines and comment lines starting with "#" + are allowed. Command files make it easy to design and debug a scenario, + then execute it multiple times (often while modifying code in-between). + After a command file is executed (without encountering an "exit" command), + the program drops into its normal keyboard input mode (which is terminated + by entering "exit"). + +The format of the fsidxupd command is: + + fsidxupd [-d] [ ...] + +where: + -d: prints lots of debugging messages during processing + fullindex: a file containing a full index from fssim + incrindexes: a list of files containing incremental indexes from fssim + +The first file name must identify a full index created by fssim. The second +and subsequent names, if present, must identify incremental indexes created +sequentially by fssim. The output of the program (currently) has a hard-coded +file name of upd-full.xml + +fssim Commands +============== + +The fssim commands are closely modeled after the Unix shell commands that +perform the same function; however, they are often greatly simplified versions +(the goal was to implement a simulator, not re-write bash). + +Unrecognized commands or commands with an invalid number of operands will +return a "** Invalid" message. Commands that do not execute successfully +(for any reason) will return the message "** Failed". That is the extent of +error messages. (This is only a test program.) + +The command prompt indicates the current working directory (as it typically +does in bash), and starts at root ("/"). File and directory names can be +entered as full paths or relative to the current directory; however, the only +place that ".." is recognized is as the single operand of a "cd" command, and +the only place that "." is recognized is as the single target for a mv or cp +command. The directory separator character is "/". The command prompt +environment supports command recall and editing (has been tested on Linux and +Mac). + +The command processor respects strings enclosed in double quotes ("). +Quoted strings can be used to create object names with embedded blanks or +to echo strings with arbitrary blank spaces. For example: + +echo Hello > "file 1" +echo " world!" >> "file 1" +cat "file 1" + +The "help" command will print a short description of all of the commands. +Here is a slightly more detailed command summary (square brackets indicate +optional parameters, curly braces indicate a choice): + +cat name - write a file's contents to the screen +cd [dirname] - change current directory; with no operands, changes to root +cls - clear the screen (tested in Linux terminal window) +cp [-r] name name - copy a file from one name to another; with -r option, + copy directory contents recursively +dir [-a] [name] - an alias for ls +echo [data] [{>|>>} name] - write data to the screen or to a file +fsck - verify that all object flags have been correctly reset +index [{-i|-f}] fileprefix - write full and/or incremental indexes to files + using names prefixed with the specified prefix (see below) +ll [name] - list long: an alias for ls -a +log [-t] - list current update log entries (optionally in time order) +ls [-a] [name] - list file or directory info (see more info below) +md dirname [...] - an alias for mkdir +mkdir dirname [...] - create directories or full directory paths +mv name name - move/rename a file or directory +rd dirname - an alias for rmdir +rm filename - remove a file +rmdir dirname - remove a directory (if it is empty) +touch name [...] - update the timestamp of existing files or directories, + or create new files (with directory paths if needed) +tree [-o] - show a tree view of the entire file system; with -o + option, also include oids + +The ls (dir) command prefixes each file or directory with some attributes in +the order "dmn". A "d" in the first position indicates a directory. An "m" +in the second position indicates that the file or directory has been modified +since the last index was written. An "n" in the third position indicates that +the directory is new since the last index (and thus must be written in its +entirety with all of its children to an incremental index). Following the +attributes are the object id, the modification time, and the object size. For +files, the size is the number of data bytes in the file; for directories, it +is the number of children in the directory. When the object of the "ls" +command is a directory, the "-a" (all) option will also show an entry (".") +which lists the status of the directory itself. (The ll command is shorthand +for "ls -a".) + +The tree command shows a full file system tree view; using the "-o" option +shows the object ids (oids) following the object names, as in this example: + + / (0) + |-file1 (1) + |=dir1 (2) + |=dir2 (4) + |-file2 (5) + |=dir3 (3) + +As can be seen from the example, a "-" in front of a name in the tree view +indicates a file, while a "=" indicates a directory. Within a directory +files are listed first followed by subdirectories, in alphabetical order. + +The mkdir (md), touch, and echo commands will create any missing lower-level +directories required in the directory or file path. + +A note using on the cp command with the "-r" option (recursive copy). This +command option requires two existing directory names as input; it copies the +*contents* of the first directory to the second, recursively. Thus, using a +directory structure such as that shown in the output of "tree" above, the +following command: + + cp -r dir1/dir2 dir3 + +would copy only file2 to dir3. In order to end up with dir2 and file2 under +dir3, one could use either + + cp -r dir1 dir3 + +or + md dir3/dir2 + cp -r dir1/dir2 dir3/dir2 + +Note, however, that the first option will copy everything under dir1 to dir3; +if dir1 had files or directories other than dir2 under it, it would copy +everything and would not give the desired results. + +Indexes +======= + +By default, the index command writes both a full index and an increment index +to a pair of files whose names start with the specified prefix. For example, +the command "index t0" will create files "t0-full.xml" and "t0-incr.xml" in +the user's current directory. (The rationale for writing both indices is that +that one can start from any full index written, apply any number of subsequent +incremental indexes they wish, and the result should be the same as the full +index corresponding to the last incremental index used.) The index command +allow overriding the default behavior by using the "-i" (incremental index +only) or "-f" (full index only) options. To simulate normal system operation, +(and minimize the number of files produced during testing), one could specify +"-f" for the index produced at the start of the run (e.g., "index -f t0"), then +use "-i" for all subsequent indexes up to the last one, and use neither flag +for the final index. This would give a starting and ending full index, and a +full set of incremental indexes to validate against the final full index. + +When testing indexes it's probably a good idea to start a command file with +the command "index [-f] t0" (or the prefix of your choice); this creates a +starting point for incremental indexes from the beginning of the session. +index commands can be interspersed in a command stream at any point, and the +resulting full and incremental indexes can be inspected (and processed +using fsidxupd). + +The fsck command searches the entire file system for any files or directories +with New or Modified flags. This can be used after an index command with the +"-i" (incremental only) flag to verify that all objects with these flags have +been visited, and that all such flags have been correctly reset. + +A pair of "t0" indexes (created at startup) looks like this: + + + 0 + + + + + + + 0 + + + + + +The two are essentially identical, since the root directory is the only one in +existence, and it is new and thus shows up in the incremental index as well. +(Actually, the t0 incremental index is useless, but it helps illustrate how +indexes work.) + +Invoking fssim with this set of commands (in a command file): + + index t0 + mkdir Dir1 + touch Dir2/Dir3/foo + touch Dir2/Dir3/bar + index t1 + touch Dir2/Dir3/baz + rmdir Dir1 + index t2 + +produced the following two "t2" indexes: + + + 0 + + + + Dir2 + 2 + + + + Dir3 + 3 + + + + 5 + bar + + + + 6 + baz + + + + 4 + foo + + + + + + + + + + + + + + Dir1 + + + + Dir2 + + + Dir3 + + + + baz + 6 + + + + + + + + + +Using fsidxupd +============== + +fsidxupd is used to verify the correctness of incremental indexes and their +application to a full index. For example, using the sample list of commands +in the Index section above, three sets of indexes will be created by fssim +when running the commands; the file names will be t0-full.xml, t0-incr.xml, +t1-full.xml, t1-incr.xml, t2-full.xml, and t2-incr.xml. The following +fsidxupd command will take the starting full index and apply the two appro- +priate incremental indexes: + + fsidxupd t0-full.xml t1-incr.xml t2-incr.xml + +The output of this command will be a file named upd-full.xml. In this example, +if the system is working correctly upd-full.xml will be identical to +t2-full.xml. In Unix-like systems, the command: + + diff upd-full.xml t2-full.xml + +would show no output (no differences). + +Program Organization +==================== + +The file fscommon.py contains all of the file system logic. It contains +object definitions for directory and file objects, and implements all of +the low-level logic for managing those objects. (In function, it roughly +corresponds to the vfs module in a Unix-like OS.) It also contains the +the python version-agnostic print function (fsprt). + +The file fsglobals.py defines global variables that must be shared between +the different modules. In order to provide a consistent naming reference +to these shared global variables, they must be in a separate python module +and that module must be imported with the same name in each program. (For +succinctness, I import it as "g", so that global values are referenced as +g.variablename.) + +The file fssim is the command processor for user file system commands; it +processes commands from a command file and/or the command line. (In Unix +terms, it implements the user shell and shell commands.) + +The file fsidxupd is a utility program that reads, updates, and writes +indexes. It uses file system functions from fscommon.py to update a full +index using data from incremental indexes. + +fsruntest +========= + +The programs also include a bash shell script to automate testing. The +format of the command is: + + fsruntest [-d] + +The command file is used to run the test. It may end with an "exit" command, +or the exit command can be entered manually while the test is running. The +"-d" option runs fssim and fsidxupd with debugging messages enabled. + +The script first invokes fssim, passing in the name of the command file. It +then invokes fsidxupd to apply all of the subsequent incremental indexes to +the first full index file created. Finally, it compares the output of fsidxupd +to the final full index created by fssim. If the output of diff is not empty, +the output is preceded by a highlighted error line (so that it stands out or +can be searched for in a large test run). + +Since it is a bash shell script, the test script runs on both Linux and Mac +systems, and with the proper Linux subsystem configured perhaps even on +Windows 10 (or earlier, using Cygwin). + + +Final Comments +============== + +The oids in the files and directories correspond to the fileuid in LTFS. +Initially I did not believe that oids (and thus fileuids) were important +to processing incremental indexes. It turns out that I was wrong, and that +a unique identifier for an object is important to the correct application +of an incremental index in one important case. Timestamps are the last +object modification time. I think all other object fields are obvious. + +The programs have not been intensively debugged. There may be lingering +errors in the commands and/or in index generation. It all seems to work, +but if you find errors (or even think you *may* have found an error) please +let me know. The better debugged the programs are, the more reliable our +results will be. + + -David Pease - pease@coati.com diff --git a/contrib/fssim/src/fscommon.py b/contrib/fssim/src/fscommon.py new file mode 100644 index 00000000..4df006c1 --- /dev/null +++ b/contrib/fssim/src/fscommon.py @@ -0,0 +1,713 @@ +# +# File System Simulator for LTFS Common Routines +# +# This module is imported into fssim and fsidxupd, and provides common +# values, objects, and functions used by both programs. +# +# (c) David Pease - IBM Almaden Research Center - May 2018 +# (c) David Pease - pease@coati.com - Sept. 2020 +# + +from copy import copy as pyobjcopy +from datetime import datetime +import fsglobals as g + +useNextOID = None # constant for readability +Log = {} # dictionary to hold update log entries + +########################################################################### +# object flag bit values # +########################################################################### + +modified = 0x01 # object has been modified since last index +new = 0x02 # directory is newly created since last index + +########################################################################### +# Directory object class # +########################################################################### + +class Dir(object): + def __init__(self, parent, oid=None, time=None): + self.oid = oid + self.flags = new + if time: # local code to avoid setting modified attribute + self._modTime = time + else: + self._modTime = str(datetime.now()) + self.parent = parent + self.contents = {} + @property + def oid(self): + return self._oid + @oid.setter + def oid(self, oid): + if oid is not None: + self._oid = oid + else: + self._oid = g.nextoid + g.nextoid += 1 + @property + def parent(self): + return self._parent + @parent.setter + def parent(self, parent): + self._parent = parent + @property + def modTime(self): + return self._modTime + @modTime.setter + def modTime(self, time=None): + if time: + self._modTime = time + else: + self._modTime = str(datetime.now()) + self.isModified = True + @property + def isDir(self): + return True + @property + def isFile(self): + return False + @property + def isModified(self): + return bool(self.flags & modified) + @isModified.setter + def isModified(self, ismodified): + if ismodified: + self.flags |= modified + else: + self.flags &= (0xff - modified) + @property + def isNew(self): + return bool(self.flags & new) + @isNew.setter + def isNew(self, isnew): + if isnew: + self.flags |= new + else: + self.flags &= (0xff - new) + @property + def children(self): + return self.contents.keys() + def contains(self, name): + return name in self.contents + def obj(self, name): + if not name in self.contents: + return None + return self.contents[name] + def addObj(self, name, obj, update=True): + self.contents[name] = obj + if update: + self.modTime = str(datetime.now()) + self.isModified = True + return True + def replObj(self, name, obj): + if name not in self.contents: + return False + self.contents[name] = obj + return True + def rmObj(self, name, verifyEmpty=True, update=True): + if not name in self.contents: + return False + obj = self.contents[name] + if obj.isDir and verifyEmpty: + if len(obj.contents) > 0: + return False + del self.contents[name] + if update: + self.modTime = str(datetime.now()) + self.isModified = True + return True + def __str__(self): + flags = ["new," * self.isNew, "mod," * self.isModified] + str = "Dir: oid %i, flags: '%s', parent:%i, size:%i %s %s" % \ + (self.oid, "".join(flags)[:-1], self.parent.oid, len(self.contents), + self.modTime[5:16], repr(self).split()[-1][2:-1]) + return str + +########################################################################### +# File object class # +########################################################################### + +class File(object): + def __init__(self, parent, oid=None, time=None): + self.oid = oid + self.flags = modified + self.modTime = time + self.parent = parent + self.data = "" + @property + def oid(self): + return self._oid + @oid.setter + def oid(self, oid): + if oid is not None: + self._oid = oid + else: + self._oid = g.nextoid + g.nextoid += 1 + @property + def parent(self): + return self._parent + @parent.setter + def parent(self, parent): + self._parent = parent + @property + def modTime(self): + return self._modTime + @modTime.setter + def modTime(self, time=None): + if time: + self._modTime = time + else: + self._modTime = str(datetime.now()) + self.isModified = True + @property + def isDir(self): + return False + @property + def isFile(self): + return True + @property + def isModified(self): + return bool(self.flags & modified) + @isModified.setter + def isModified(self, ismodified): + if ismodified: + self.flags |= modified + else: + self.flags &= (0xff - modified) + @property + def data(self): + return self._data + @data.setter + def data(self, data): + self._data = data if data else "" + def __str__(self): + flag = "mod" * self.isModified + str = "File: oid %i, flags:'%s', parent:%i, size:%i %s %s" % \ + (self.oid, flag, self.parent.oid, len(self.data), + self.modTime[5:16], repr(self).split()[-1][2:-1]) + return str + +########################################################################### +# python version-agnostic print function # +########################################################################### + +def fsprt(*args): + outstr = "" + for arg in args: + outstr += (str(arg) + " ") + print(outstr) + +########################################################################### +# debugging-only print function # +########################################################################### + +def dbprt(*args): + if g.debug: + fsprt(*args) + +########################################################################### +# create a log entry # +########################################################################### + +def log(name, oid, reason): + logkey = name + "#" + str(oid) + # don't overwrite a "New" log entry with a "Mod" one for the same object + if logkey in Log and Log[logkey][0] == "New": + if reason == "Mod": + dbprt("Skip log: Mod for", name, "- is already New") + return + # trim unneeded log entries if parents are New or Deleted + # (processing log entries in reverse time order - why does it matter??) + for entkey in sorted(Log, key=lambda entkey: Log[entkey][1], reverse=True): + entpath, entoid = entkey.rsplit("#", 1) + entreason, enttime = Log[entkey] + # if a parent dir is already "New", don't bother to log updates + if entreason == "New" and name.startswith(joinPath(entpath,"/")): + dbprt("Skip log", reason, "for", name, "- parent is New") + return + # if this is a parent being deleted, remove unneeded entries for children + elif reason == "DelD": + if entpath.startswith(joinPath(name, "/")): + dbprt("Remove log", entreason, "for", entpath, "- parent Deleted") + del Log[entkey] + Log[logkey] = (reason, datetime.now()) + +########################################################################### +# display log entries in pathname (processing) or time order # +########################################################################### + +def printLog(timeOrder=False): + if timeOrder: + sortkey = lambda k: Log[k][1] + else: + sortkey = None + for k in sorted(Log, key=sortkey): + if timeOrder: + fsprt(str(Log[k][1])[:-7], Log[k][0], k) + else: + fsprt("%-20s" % k, Log[k][0], str(Log[k][1])[:-7]) + return True + +########################################################################### +# file system common utility functions # +########################################################################### + +# return a fully-qualified name (with no trailing slash) +def fullName(name): + dbprt("fullName in:", name) + if name.startswith("/"): + fullname = name + else: + fullname = joinPath(g.curnm, name) + if len(fullname) > 1 and fullname.endswith("/"): + fullname = fullname[:-1] + dbprt("fullName out:", fullname) + return fullname + +# return full path and name portions of a filespec +def splitPath(name): + path,name = fullName(name).rsplit("/",1) + if not path: + path = "/" + return path,name + +# return a properly joined path and file name +def joinPath(path, name): + if name.startswith("/"): + name = name[1:] + if path.endswith("/"): + return path + name + else: + return path + "/" + name + +# process a directory path, output depends on parentRef value: +# False returns: full name of object, object instance (default behavior) +# True returns: full path to object, parent object instance, object name +def dirpath(name, parentRef=False): + fullname = fullName(name) + if not parentRef and fullname != "/": + fullname += "/" + dbprt("dirpath in:", fullname) + wdir = g.root + wdnm = "/" + parts = fullname[1:].split("/") + if parts[-1]: + lastpart = len(parts)-1 + else: + lastpart = len(parts)-2 + for cnt, dn in enumerate(parts[:-1]): + if wdir.contains(dn): + if wdir.obj(dn).isDir or cnt == lastpart: + wdir = wdir.obj(dn) + wdnm = joinPath(wdnm, dn) + continue + dbprt("dirpath out: None") + if parentRef: + return None, None, None + else: + return None, None + dbprt("dirpath out:", wdnm, wdir.oid, parts[-1]) + if parentRef: + return wdnm, wdir, parts[-1] + else: + return wdnm, wdir + +# depth-first-search of a file system tree with function invocation on descent +# parameters: +# sdobj - starting directory object (top of dfs search tree) +# sdnm - the fully-qualified name of the starting directory +# func - the function to be invoked while traversing the tree +# params - a tuple of parameters to pass to func +# return value: +# boolean - from "enter" into a directory, flag indicating whether to +# descend into (that is process the members of) that directory, +# otherwise flag from func that halts processing of dir if False +# parameters passed to func: +# reason - one of "enter", "exit", "file", or "dir" +# sdobj, sdnm, object name if "file" or "dir" else None, params +def dfs(sdobj, sdnm, func, params): + if sdnm == "/": + name = "/" + else: + name = splitPath(sdnm)[1] + descend = func("enter", sdobj, sdnm, None, params) + if descend: + for name in sorted(sdobj.children, # process files before subdirs + key=lambda name: 'x\ff'+name if sdobj.obj(name).isDir else name): + if sdobj.obj(name).isFile: + if not func("file", sdobj, sdnm, name, params): + return False + else: + if not func("dir", sdobj, sdnm, name, params): + return False + wdir = sdobj.obj(name) + wdnm = joinPath(sdnm, name) + if not dfs(wdir, wdnm, func, params): + return False + return func("leave", sdobj, sdnm, None, params) + else: + return True + +# create a new directory or file obj, add it to parent, return object instance +def makeObj(name, dir=False, oid=None, ts=None, update=True): + path, parent, newname = dirpath(name, parentRef=True) + if parent == None: + return None + if parent.contains(newname): + return None + if dir: + newobj = Dir(parent, oid=oid, time=ts) + state = "New" + else: + newobj = File(parent, oid=oid, time=ts) + state = "Mod" + log(joinPath(path, newname), newobj.oid, state) + parent.addObj(newname, newobj, update=update) + return newobj + +# remove a directory or file +def rmObj(name, dir=False, update=True): + if name == "/": + return False # can't remove root + path, parent, oldname = dirpath(name, parentRef=True) + if parent == None or not parent.contains(oldname): + return False + obj = parent.obj(oldname) + if dir: + if obj.isFile: + return False + if obj == g.curdir: + return False # can't remove current dir + else: + if obj.isDir: + return False + if not parent.rmObj(oldname, update=update): # for dir, parent will if empty + return False + deltype = "DelD" if dir else "DelF" + log(joinPath(path, oldname), obj.oid, deltype) + return True + +# print info about an object, including memory address (for debugging only) +def printObj(obj, prefix="", children=False): + fsprt(prefix+str(obj)) + if children and obj.isDir: + for child in obj.children: + fsprt(" "+str(obj.obj(child))) + +# move or rename a directory or file, or copy a single file +def movecopy(srcname, tgtname, copy=False): + # get full paths of source and target + if srcname == "/": # can't move root, copying it would be recursive + return False + if tgtname == ".": + tgtname = g.curnm + fulltgt = tgtname # save original name for later + # get source and target object and name information + srcpath, srcobj = dirpath(srcname) + tgtpath, tgtparent, tgtname = dirpath(tgtname, parentRef=True) + if not srcobj or not tgtparent: + return False + # don't allow recursive copy + if tgtpath.startswith(srcpath+"/"): + return False + # cannot copy a complete directory + if copy and srcobj.isDir: + return False + dbprt("source:", srcpath, srcobj.oid) + dbprt("target:", tgtpath, tgtparent.oid, tgtname) + srcname = splitPath(srcpath)[1] + # if target is simply "/", must have a file name to continue + if not tgtname: + tgtname = srcname + # if target is a directory, make it new parent and copy source filename + if tgtparent.contains(tgtname) and tgtparent.obj(tgtname).isDir: + tgtpath, tgtparent = dirpath(fulltgt) + tgtname = srcname + dbprt("new target info:", tgtpath, tgtname, tgtparent.oid) + # now that we have all the object info, make sure not moving object to self + srcparent = srcobj.parent + if srcparent == tgtparent and srcname == tgtname: + dbprt("Move to self!") + return False + # if target already exists, decide what needs to be done (quite complex!) + remtobj = None # flag to delete old target object + if tgtparent.contains(tgtname): + # if the target is a directory + if tgtparent.obj(tgtname).isDir: + # and the source is a file + if srcobj.isFile: + # if target contains new object name, it must be a file + if tgtparent.obj(tgtname).contains(srcname): + if tgtparent.obj(tgtname).obj(srcname).isDir: + return False + # it's a file, so we must remove it + else: + remtobj = tgtparent.obj(tgtname).obj(srcname) + # the target is a file + else: + # if the source is a directory, can't move it to a file + if srcobj.isDir: + return False + # source is a file + if not copy: + remtobj = tgtparent.obj(tgtname) + dbprt("tgt exists:", ("remove "+str(remtobj.oid)) if remtobj else "") + # we're finally ready to move something + dbprt("mvcp:", srcpath,srcname,srcobj.oid, tgtpath,tgtname,tgtparent.oid) + # if copy, create a copy of the source object + if copy: + newobj = pyobjcopy(srcobj) # new python object, copy of srcobj + newobj.oid = useNextOID + else: # move (move original object, make copy in source for deletion) + newobj = srcobj # newobj is srcobj (same python object) + srcobj = pyobjcopy(newobj) # create new python object copy for srcobj + if not srcparent.replObj(srcname, srcobj): # replace srcobj ref in parent + return False + # set target object's parent and state + newobj.parent = tgtparent + if newobj.isDir: + newobj.isNew = True + status = "New" + else: + newobj.isModified = True + status = "Mod" + # if necessary, remove existing file that is being replaced + if remtobj: + remoid = tgtparent.obj(tgtname).oid + if not tgtparent.rmObj(tgtname): + return False + log(joinPath(tgtpath, tgtname), remoid, "DelF") + # put moved/copied object in its new home + tgtparent.addObj(tgtname, newobj) # put object under target name + log(joinPath(tgtpath, tgtname), newobj.oid, status) + # for move, remove object from its old home + if not copy: + if not srcparent.rmObj(srcname, verifyEmpty=False): # rem src from parent + fsprt("Error: cannot remove source object for move; object copied.") + return False + deltype = "DelF" if srcobj.isFile else "DelD" + dbprt("Creating log:", deltype, srcpath, srcobj.oid) + log(srcpath, srcobj.oid, deltype) + if g.debug: + printObj(newobj, "mvcp new: ") + printObj(srcobj, "mvcp src: ") + return True + +# copy a directory's contents to an existing directory recursively +def copyrecurs(inloc, outloc): + # get source and target object and name information + if outloc == ".": + outloc = g.curnm + srcpath, srcobj = dirpath(inloc) + tgtpath, tgtobj = dirpath(outloc) + if not srcobj or not tgtobj: + return False + # don't allow recursive copy + if tgtpath.startswith(srcpath+"/"): + return False + # both source and target must be existing directories + if not srcobj.isDir or not tgtobj.isDir: + return False + # recursively copy files and directories + return dfs(srcobj, srcpath, copyObj, (srcpath, tgtpath)) + +# make a copy of a single file or directory (invoked through dfs()) +def copyObj(desc, dirobj, path, name, moreparams): + srcfname, tgtfname = moreparams + if desc not in ("file", "dir"): + return True + dbprt("copyObj:", desc, dirobj.oid, path, name, srcfname, tgtfname) + addpath = path[len(srcfname):] + if addpath: + tgtdir = tgtfname + addpath + else: + tgtdir = tgtfname + newname = joinPath(tgtdir, name) + if desc == "file": + oldname = joinPath(path, name) + dbprt("Copying file", oldname, "to", newname) + return movecopy(oldname, newname, copy=True) + elif desc == "dir": + dbprt("Creating directory", newname) + if not makeObj(newname, dir=True): + return False + # should copy other attributes from source to new dir here, if needed + return True + +########################################################################### +# routines used in generating indexes # +########################################################################### + +# write a full index to a file (uses dfs to descend entire tree) +def fullIndex(fn): + global indexLevel, Log + fullfn = fn + "-full.xml" + with open(fullfn, "w") as fh: + fh.write("\n") + indexLevel = 0 # used only for indentation of xml + indexEnt("dir", g.root, "/", None, ("full", fh)) + result = dfs(g.root, "/", indexEnt, ("full", fh)) + fh.write("\n") + Log = {} # clear log after writing an index + if not result: + fsprt("Error occurred creating full index") + return result + +# write an incremental index to a file +def incrIndex(fn): + global indexLevel, Log + success = True + incrfn = fn + "-incr.xml" + with open(incrfn, "w") as fh: + fh.write("\n") + indexLevel = 0 # used for indentation and to close paths + # always start with an entry for the root dir, whether or not it's in log + indexEnt("dir", g.root, "/", "", ("incr", fh)) + indexLevel += 1 + lastpath = "/" + pathindir = None + # process all log entries sorted in name,oid order + for logKey in sorted(Log): + # get full name, oid, log entry type and timestamp for this log entry + fullname, oid = logKey.rsplit("#", 1) + if fullname == "/" and oid == "0": # we already did the root dir + continue + path, name = splitPath(fullname) + reason, ts = Log[logKey] + # if debugging, show log entry + dbprt("Log entry: %s (%i) %s %s\n" % (fullname, int(oid), reason, ts)) + # get object reference for log target + obj = dirpath(fullname)[1] # already have full name, need obj + # didn't find object for this entry, error unless it was for deletion + if not obj and not reason.startswith("Del"): + dbprt("Object matching log entry not found", fullname, oid) + success = False + break + # found the same name but a different oid, also bad unless delete + if not reason.startswith("Del") and obj.oid != int(oid): + dbprt("Object oids do not match:", fullname, oid, obj.oid) + continue + # close out unneeded directories from prior object + lastdirs = lastpath.split("/")[1:] if lastpath != "/" else [] + newdirs = path.split("/")[1:] if path != "/" else [] + newLevel = len(newdirs) + lastmatch = -1 + for i in range(min(len(lastdirs), len(newdirs))+1): + if len(lastdirs) > i and len(newdirs) > i: + if lastdirs[i] == newdirs[i]: + continue + lastmatch = i-1 + break + i = 0 # inialize in case following loops null + for i in range(len(lastdirs)-1, lastmatch, -1): + fh.write(" " * (i+1) + " \n") + fh.write(" " * (i+1) + "\n") + # build new directory path structure as needed for object + for i in range(lastmatch+1, newLevel): + prefix = " " * (i+1) + temppath = "/".join(['']+newdirs[:i+1]) # full path name of dir + if temppath == pathindir: # if we just added this dir + continue # to the index, skip it + fh.write(prefix + "\n") + fh.write(prefix + " " + newdirs[i] + "\n") + tempobj = dirpath(temppath)[1] + if tempobj.isModified: # parent dir may have updated modtime + fh.write(prefix + " \n") + tempobj.isModified = False + fh.write(prefix + " \n") + indexLevel = newLevel + 1 + pathindir = None + # process Deletes here rather than using indexEnt() + if reason.startswith("Del"): + lastpath = path # make sure to save last path for deletes as well + if obj: + if obj.oid == int(oid): + dbprt("Found deleted object in file system", fullname, oid) + success = False + break + else: + continue # don't delete object if it's been recreated + prefix = " " * indexLevel + indextype = "file" if reason == "DelF" else "directory" + fh.write(prefix + "<" + indextype + ">\n") + fh.write(prefix + " " + name + "\n") + fh.write(prefix + " \n") + fh.write(prefix + "\n") + continue + # write the object entry into the index + newdir = obj.isDir and obj.isNew # indexEnt() will reset this + indextype = "file" if obj.isFile else "dir" + indexdirobj = obj.parent if obj.parent else g.root + indexEnt(indextype, indexdirobj, fullname, name, ("incr", fh)) + # if object was a New directory, output its full subtree + if obj.isDir: + if newdir: + if not dfs(obj, fullname, indexEnt, ("full", fh)): + dbprt("Error occurred in dfs for incremental index") + success = False + break + else: + pathindir = joinPath(path, name) # this feels like a kludge! + # save index state from this entry to compare to the next + lastpath = path + # close out the directory tree and the incremental index + for i in range(indexLevel-1, 0, -1): + fh.write(" " * i + " \n") + fh.write(" " * i + "\n") + fh.write(" \n") + fh.write("\n") + # clear the log and return + Log = {} + if not success: + # delete incremental index file, but for now keep it for debugging + fsprt("Error occurred creating incremental index") + return success + +# write the XML for single index entry (this function my be invoked directly, +# but is often invoked through dfs(), thus the use of params) +def indexEnt(desc, dirobj, path, name, params): + global indexLevel + enttype, fh = params + dbprt("Index entry:", desc, dirobj.oid, path, name, enttype) + if not name: # entering or leaving a directory, or root dir + obj = dirobj # obj is the directory itself + if path != "/": + name = splitPath(path)[1] + else: + obj = dirobj.obj(name) # obj is a member of the directory + if desc == "enter": + indexLevel += 1 + elif desc == "file": + prefix = " " * indexLevel + fh.write(prefix + "\n") + fh.write(prefix + " " + name + "\n") + fh.write(prefix + " " + str(obj.oid) + "\n") + # note: if enttype == "incr", should only output changed attributes + fh.write(prefix + " \n") + if obj.data or enttype == "incr": # if incr, replace any old data + fh.write(prefix + " " + obj.data + "\n") + fh.write(prefix + "\n") + obj.isModified = False + elif desc == "dir": + prefix = " " * indexLevel + if obj != g.root: + fh.write(prefix + "\n") + fh.write(prefix + " " + name + "\n") + if enttype == "full" or obj.isNew: # ??? does this match spec? + fh.write(prefix + " " + str(obj.oid) + "\n") + fh.write(prefix + " \n") + fh.write(prefix + " \n") + obj.isModified = False + obj.isNew = False + elif desc == "leave": + indexLevel -= 1 + prefix = " " * indexLevel + if enttype == "full" or obj != g.root: + fh.write(prefix + " \n") + if obj != g.root: + fh.write(prefix + "\n") + return True + diff --git a/contrib/fssim/src/fsglobals.py b/contrib/fssim/src/fsglobals.py new file mode 100644 index 00000000..12d1d84b --- /dev/null +++ b/contrib/fssim/src/fsglobals.py @@ -0,0 +1,10 @@ +# +# fssim, fsidxupd, fscommon shared global variables +# + +debug = False # debugging mode flag +nextoid = 0 # master object id +root = None # root directory object +curdir = None # current directory object +curnm = None # current directory path string + diff --git a/contrib/fssim/src/fsidxupd b/contrib/fssim/src/fsidxupd new file mode 100755 index 00000000..06991f42 --- /dev/null +++ b/contrib/fssim/src/fsidxupd @@ -0,0 +1,159 @@ +#!/usr/bin/python3 +# +# File System Index Processor for LTFS +# +# This program reads in a full index created by the fssim program, then +# updates that index using one or more incremental indexes created +# sequentially by the same program. At the end, it writes the resulting +# xml index to a file (upd-full.xml). +# +# Usage: +# +# fsidxupd [-d] [ ...] +# +# where: +# -d: prints lots of debugging messages during processing +# fullindex: a file containing a full index (from fssim) +# incrindex: a file containing an incremental index (from fssim) +# +# (c) David Pease - IBM Almaden Research Center - May 2018 +# (c) David Pease - pease@coati.com - Sept. 2020 +# + +from sys import argv, exit +from xml.etree.ElementTree import parse +from fscommon import * +import fsglobals as g + +########################################################################### +# process xml children (files and dirs) in blocks recursivley # +########################################################################### + +def processContents(path, contents, full=False): + + for child in contents: + values = {} + for val in child: + if val.tag == "contents": + values[val.tag] = val + else: + values[val.tag] = val.text + name = joinPath(path, values["name"]) + dir = child.tag == "directory" + if full: + obj = makeObj(name, oid=values["oid"], ts=values["time"], dir=dir, + update=False) + if not obj: + return False + if "data" in values: + obj.data = values["data"] + else: # is incremental + fullpath, obj = dirpath(name) + if not obj: + if "deleted" in values: + pass # already deleted, so okay + else: + dbprt("Creating:", name, values) + if not "oid" in values: + fsprt("Error creating %s, no oid in index" % name) + return False + obj = makeObj(name, oid=values["oid"], ts=values["time"], + dir=dir, update=False) + if not obj: + return False + if "data" in values: + obj.data = values["data"] + else: + if "deleted" in values: + obj.parent.rmObj(values["name"], update=False,verifyEmpty=False) + else: + if "oid" in values and obj.oid != values["oid"]: + if not obj.parent.rmObj(values["name"], update=False, + verifyEmpty=False): + return False + obj = makeObj(name, oid=values["oid"], ts=values["time"], + dir=dir, update=False) + if not obj: + return False + else: + if "time" in values: + obj.modTime = values["time"] + if "data" in values: + obj.data = values["data"] + if dir and not "deleted" in values: + dbprt("Entering:", child.tag, values) + if not processContents(name, values["contents"], full=full): + return False + return True + +########################################################################### +# main line # +########################################################################### + +# do some parameter validation +if len(argv) > 1 and argv[1] == "-d": + g.debug = True + del argv[1] +if len(argv) < 2: + fsprt("Too few parameters") + exit(12) +fullfn = argv[1] +incrlist = argv[2:] + +# first build the base filesystem from the full index +try: + xml = parse(fullfn) +except: + fsprt("Error parsing XML in file", fullfn) + exit(12) +xmlroot = xml.getroot() +if xmlroot.tag != "fullindex": + fsprt("Invalid full index file header", xmlroot.tag) + exit(12) +oid = ts = contents = None +for child in xmlroot: + if child.tag == "oid": + oid = int(child.text) + elif child.tag == "time": + ts = child.text + elif child.tag == "contents": + contents = child + else: + fsprt("Invalid tag", tag) +if oid == None or not ts or contents is None: + fsprt("Missing root directory entries") + exit(12) +g.root = Dir(None, oid=oid, time=ts) +g.curdir = g.root +g.curnm = "/" +if not processContents("/", contents, full=True): + fsprt("Error creating full index structure.") + exit(12) + +# now start applying updates from incremental indexes +for incrfn in incrlist: + try: + xml = parse(incrfn) + except: + fsprt("Error parsing XML in file", incrfn) + exit(12) + xmlroot = xml.getroot() + if xmlroot.tag != "incrementalindex": + fsprt("Invalid incremental index file header", xmlroot.tag) + exit(12) + ts = contents = None + for child in xmlroot: + if child.tag == "time": + ts = child.text + elif child.tag == "contents": + contents = child + if ts: + g.root.modTime = ts + if contents is not None: + if not processContents("/", contents): + fsprt("Error applying incremental index", incrfn) + exit(12) + +# write out final results as a full index +fullIndex("upd") + diff --git a/contrib/fssim/src/fsruntest b/contrib/fssim/src/fsruntest new file mode 100755 index 00000000..4f9e0e79 --- /dev/null +++ b/contrib/fssim/src/fsruntest @@ -0,0 +1,74 @@ +#!/bin/bash +# +# Run an fssim test case, apply incremental indexes to initial full index, +# compare result to final full index, and display any difference +# + +# process operands and set values +if [ $# -lt 1 ]; then + echo "Usage:" $0 "[-d] " + exit 1 +fi +if [ $1 == "-d" ]; then + OPT="-d" + shift 1 +else + OPT="" +fi +if [ $# -ne 1 ]; then + echo "Missing command file name" + exit 1 +fi +if [ ! -f $1 ]; then + echo "File" $1 "does not exist" + exit 1 +fi + +# initialize xml file name suffixes +FNAME="-full.xml" +INAME="-incr.xml" + +# create list of indexes written by input command file +FLIST="" +while read -r prefix; do + FLIST="$FLIST $prefix" +done < <(grep "^ *index" $1 | rev | cut -d" " -f 1 | rev) + +# verify that at least two indexes are written by test case +alist=( $FLIST) +if [ ${#alist[@]} -lt 2 ]; then + echo "Too few indexes written in test script $1 for comparison" + exit 1 +fi + +# build file name strings based on list of indexes and name suffixes +FULLFILE="" +INCRFILES="" +for prefix in $FLIST; do + if [[ $FULLFILE == "" ]]; then + FULLFILE=$prefix$FNAME + else + INCRFILES="$INCRFILES $prefix$INAME" + COMPFILE=$prefix$FNAME + fi +done + +# build the actual command lines +CMD1="./fssim $OPT $1" +CMD2="./fsidxupd $OPT $FULLFILE $INCRFILES" +CMD3="diff upd-full.xml $COMPFILE" + +# display each command, then execute it +echo $CMD1 +$CMD1 +echo $CMD2 +$CMD2 +echo $CMD3 +$CMD3 > tempout.txt + +# check for errors and display them +if [ -s tempout.txt ]; then + echo "************** Errors from diff - $1 *****************" + cat tempout.txt +fi +rm tempout.txt diff --git a/contrib/fssim/src/fssim b/contrib/fssim/src/fssim new file mode 100755 index 00000000..ac4ff394 --- /dev/null +++ b/contrib/fssim/src/fssim @@ -0,0 +1,376 @@ +#!/usr/bin/python3 +# +# File System Simulator for LTFS +# +# This program simulates file system directory activity such as creating or +# deleteing files and directories, moving and copying files or directories, +# listing information, etc. It also writes full and incremental XML indexes +# to files on request. +# +# Usage: +# +# fssim [-d] [command file] +# +# where: +# -d: prints lots of debugging messages during processing +# command file: executes commands from specified text file at startup +# +# (c) David Pease - IBM Almaden Research Center - May 2018 +# (c) David Pease - pease@coati.com - Sept. 2020 +# + +from sys import argv, exit, version +from fscommon import * +import fsglobals as g +import readline + +TimeNow = None # a constant value for readability + +########################################################################### +# filesystem command functions # +########################################################################### + +# cd: change directory +def cd(opt, name=None): + if name == "/" or name == None: + g.curdir = g.root + g.curnm = "/" + return True + if name == ".." and g.curnm != "/": + g.curdir = g.curdir.parent + g.curnm = splitPath(g.curnm)[0] + if g.curnm == "": + g.curnm = "/" + return True + dirnm, newdir = dirpath(name) + if newdir == None: + return False + g.curdir = newdir + g.curnm = dirnm + return True + +# md: create a directory or (multi-level) directory path +def md(opt, *names): + for name in names: + fullname = fullName(name) + pieces = fullname[1:].split("/") + path = "" + for piece in pieces: + path += ("/" + piece) + fpath, obj = dirpath(path) + if not obj: + if not makeObj(path, dir=True): + return False + return True + +# rd: remove a directory +def rd(opt, name): + return rmObj(name, dir=True) + +# rm: remove a file +def rm(opt, name): + return rmObj(name) + +# cp: copy a single file, or copy a directory's contents recursively +def cp(opt, oldname, newname): + if opt: + if opt == "-r": + return copyrecurs(oldname, newname) + else: + return False + return movecopy(oldname, newname, copy=True) + +# mv: move a file or directory +def mv(opt, oldname, newname): + return movecopy(oldname, newname) + +# tf: update an object's timestamp or create a new file +def tf(opt, *names): + for name in names: + if name == "/": # special case for touching root dir + g.root.modTime = TimeNow + log(name, 0, "Mod") + continue + path, parent, objname = dirpath(name, parentRef=True) + if parent == None: + if not md(None, splitPath(name)[0]): + return False + path, parent, objname = dirpath(name, parentRef=True) + if parent == None: + return False + if not parent.contains(objname): + if not makeObj(name): + return False + else: + parent.obj(objname).modTime = TimeNow + log(joinPath(path, objname), parent.obj(objname).oid, "Mod") + return True + +# ec: echo string to a file or the screen +def ec(opt, *args): + if len(args) >= 2 and args[-2] in (">", ">>"): + name = args[-1] + path, obj = dirpath(name) + if obj: + if obj.isDir: + return False + obj.modTime = TimeNow + else: # need to create file first + if not tf(None, name): + return False + path, obj = dirpath(name) + if not obj: + return False + data = " ".join(args[:len(args)-2]) + if args[-2] == ">>": + obj.data += data + else: + obj.data = data + log(path, obj.oid, "Mod") + else: + fsprt(*args) + return True + +# ct: cat (print) a file's data +def ct(opt, name): + path, obj = dirpath(name) + if not obj or obj.isDir: + return False + fsprt(obj.data) + return True + +# ls: list directory contents or file information with optional information +def ls(opt, dirnm=None, showall=False): + if opt: + if opt == "-a": + showall = True + else: + return False + if not dirnm: + wobj = g.curdir + dirnm = g.curnm + else: + path, wobj = dirpath(dirnm) + if not wobj: + return False + if wobj.isFile: + flag = "-m-" if wobj.isModified else "---" + lth = len(wobj.data) + fsprt(flag, wobj.oid, wobj.modTime[5:16], lth, splitPath(path)[1]) + return True + if showall: + flag = "d" +"-m"[wobj.isModified] +"-n"[wobj.isNew] + fsprt(flag, wobj.oid, wobj.modTime[5:16], len(wobj.children), ".") + for name in sorted(wobj.children, # process subdirs before files + key=lambda name: "\0"+name if wobj.obj(name).isDir else name): + if wobj.obj(name).isDir: + flag = "d" + "-m"[wobj.obj(name).isModified] + \ + "-n"[wobj.obj(name).isNew] + lth = len(wobj.obj(name).children) + else: + flag = "-" + "-m"[wobj.obj(name).isModified] + "-" + lth = len(wobj.obj(name).data) + fsprt(flag, wobj.obj(name).oid, wobj.obj(name).modTime[5:16], lth, name) + return True + +# ll: list long, same as ls -a +def ll(opt, dirnm=None): + return ls(opt, dirnm, showall=True) + +# tr: show a tree view of file system with optional oids +def tr(opt): + oids = False + if opt: + if opt == "-o": + oids = True + else: + return False + if not oids: + fsprt("/") + else: + fsprt("/ (0)") + return dfs(g.root, "/", tree, oids) + +# print out one line of the tree view (invoked through dfs()) +def tree(desc, dirobj, path, name, oids): + if desc not in ("file", "dir"): + return True + if path == "/": + prefix = "" + else: + prefix = path.count("/") * " " + if oids: + oidstr = " (" + str(dirobj.obj(name).oid) + ")" + else: + oidstr = "" + if desc == "file": + fsprt(prefix + " |-" + name + oidstr) + elif desc == "dir": + fsprt(prefix + " |=" + name + oidstr) + return True + +# cl: clear the screen +def cl(opt): + fsprt("\x1b\x5b\x48\x1b\x5b\x32\x4a") + return True + +# ix: write out both a full and an incremental index +def ix(opt, fnprefix): + full = True + incr = True + if opt: + if opt == "-f": + incr = False + elif opt == "-i": + full = False + else: + return False + if incr: + result = incrIndex(fnprefix) + if not result: + full = True # if incremental fails, force full index + if full: # full index clears log, all file flags, so must be last + result = fullIndex(fnprefix) + return result + +# pl: display update log entries by name or optionally by time +def pl(opt): + byTime = False + if opt: + if opt == "-t": + byTime = True + else: + return False + return printLog(byTime) + +# fc: check file system for extraneous or incorrect information +def fc(opt): + if g.root.isNew or g.root.isModified: + fsprt("/", ("Mod" * g.root.isModified), ("New" * g.root.isNew)) + return dfs(g.root, "/", chkMods, None) + +# display objects that indicate New or Modified (invoked thru dfs()) +def chkMods(desc, dirobj, path, name, dummy): + if desc not in ("file", "dir"): + return True + obj = dirobj.obj(name) + mod = obj.isModified + new = obj.isDir and obj.isNew + if mod or new: + fsprt(joinPath(path, name), ("Mod" * mod), ("New" * new)) + return True + +########################################################################### +# execute a single input command (from command file or keyboard) # +########################################################################### + +def execCommand(input): + if input == "exit": + return False + if input == "help": + fsprt("Enter 'exit', 'help', or one of the following commands:") + for cmdname in sorted(cmdTable): + fsprt("%-5s %s" % (cmdname, cmdTable[cmdname][3])) + return True + # tokenize input string allowing quoted strings + sstr = input.split('"') + if len(sstr) % 2 != 1: + fsprt("** Invalid") + return True + input = [] + for ct, ss in enumerate(sstr): + if not ss: + continue + if ct % 2 == 1: + input.append(ss) + continue + else: + input += ss.split() + # process the command + cmdnm = input[0] + if cmdnm not in cmdTable: + fsprt("** Invalid") + return True + func = cmdTable[cmdnm][2] + opt = None + if len(input) > 1 and input[1].startswith("-"): + opt = input[1] + del input[1] + if (len(input)-1) < cmdTable[cmdnm][0] or \ + (len(input)-1) > cmdTable[cmdnm][1]: + fsprt("** Invalid") + return True + params = [] + for p in input[1:]: + params.append(p) + if not func(opt, *params): + fsprt("** Failed") + return True + +########################################################################### +# command table - name: (min operands, max operands, function, help text) # +########################################################################### + +cmdTable = { + # cmd min max fn help text + "cd": (0, 1, cd, "[dirname] - change current directory"), + "mkdir": (1,20, md, "dirname ... - make directories"), + "md": (1,20, md, "dirname ... - alias for mkdir"), + "touch": (1,20, tf, "name ... - create file(s) or update time(s)"), + "rmdir": (1, 1, rd, "dirname - remove directory"), + "rd": (1, 1, rd, "dirname - alias for rmdir"), + "rm": (1, 1, rm, "filename - remove file"), + "cp": (2, 2, cp, "[-r] name name - copy one file, or dir recursively"), + "mv": (2, 2, mv, "name name - move/rename file or directory"), + "ls": (0, 1, ls, "[-a] [name] - list file or directory info"), + "dir": (0, 1, ls, "[-a] [name] - alias for ls"), + "ll": (0, 1, ll, "[name] - alias for ls -a (ls with all info)"), + "tree": (0, 0, tr, "[-o] - show a tree view of the file system"), + "echo": (1, 20,ec, "[data] [>|>> name] - write data to screen or file"), + "cat": (1, 1, ct, "name - write a file's contents to the screen"), + "cls": (0, 0, cl, " - clear the screen"), + "index": (1, 1, ix, "[-f|-i] fileprefix - write full and/or incr indexes"), + "log": (0, 0, pl, "log [-t] - display log entries by name or time"), + "fsck": (0, 0, fc, " - check files and dirs for correctness"), +} + +########################################################################### +# main line # +########################################################################### + +# create root directory with no parent and make it the current dir +g.root = Dir(None) # equivalent to mkfs -t fssim ;-) +log("/", 0, "New") +g.curdir = g.root +g.curnm = "/" + +# set debugging flag +g.debug = "-d" in argv[1:] + +# if command file specified, execute commands from it +if len(argv) > 1 and not argv[-1].startswith("-"): + infname = argv[-1] + with open(infname, "r") as infile: + inlines = infile.readlines() + for inpt in inlines: + inpt = inpt.strip() + if inpt: + if g.debug: + fsprt("==============================") + if inpt.startswith("#"): + fsprt(inpt) + else: + fsprt(g.curnm+"> "+inpt) + if not execCommand(inpt): + exit(0) + +# go into interactive mode +fsprt('Enter command, "exit", or "help":') +while True: + if version.startswith("2"): + input = raw_input # use python 3.x function name + inpt = input(g.curnm+"> ").strip() + if inpt and not inpt.startswith("#"): + if not execCommand(inpt): + exit(0) + diff --git a/contrib/fssim/testcases/daptest.txt b/contrib/fssim/testcases/daptest.txt new file mode 100644 index 00000000..75dfaee5 --- /dev/null +++ b/contrib/fssim/testcases/daptest.txt @@ -0,0 +1,11 @@ +index t0 +touch a/b/c/foo +log +index t1 +touch a/b/bar +touch a/b/baz +ll a/b +log +index t2 +exit + diff --git a/contrib/fssim/testcases/daptest2.txt b/contrib/fssim/testcases/daptest2.txt new file mode 100644 index 00000000..7f97175f --- /dev/null +++ b/contrib/fssim/testcases/daptest2.txt @@ -0,0 +1,21 @@ +index t0 +ls +ls -a +touch a/b/c/foo +touch a/b/c/bar +log +ls -a +ls -a b +ls -a a/b +md z +ls -a +ls -a a/b/c +index t1 +ll +ll a +md c +ll +ls +rd c +index t2 +exit diff --git a/contrib/fssim/testcases/daptest3.txt b/contrib/fssim/testcases/daptest3.txt new file mode 100644 index 00000000..58144bd5 --- /dev/null +++ b/contrib/fssim/testcases/daptest3.txt @@ -0,0 +1,10 @@ +index t0 +touch foo +echo "Hi" > bar +ll +log +mv bar foo +ll +log +index t1 +exit diff --git a/contrib/fssim/testcases/daptest4.txt b/contrib/fssim/testcases/daptest4.txt new file mode 100644 index 00000000..22cfad72 --- /dev/null +++ b/contrib/fssim/testcases/daptest4.txt @@ -0,0 +1,7 @@ +index t0 +touch a/b/foo +touch a/c/foo +index t1 +touch a/c/foo +index t2 +exit diff --git a/contrib/fssim/testcases/daptest5.txt b/contrib/fssim/testcases/daptest5.txt new file mode 100644 index 00000000..130a219f --- /dev/null +++ b/contrib/fssim/testcases/daptest5.txt @@ -0,0 +1,34 @@ +index t30 +# /A +# |_ /B +# | |_ n1 +# |_ n2 +# |_ n3 +# /C +# |_ n4 +# |_ n5 +# /n1 +# /D +# /E +# +touch /A/B/n1 /A/n2 /A/n3 /C/n4 /C/n5 n1 +mkdir D E +tree +index t31 +# 1. Delete /A/B/n1 +# Rename the directory /C to /A/B/n1 +rm /A/B/n1 +mv /C /A/B/n1 +tree -o +index t32 +# 2. Append data data to /A/B/n1 +# Delete /A/B/n1 +# Rename file /n1 to /A/B/n1 +touch /A/B/n1 +rm /A/B/n1 +mv n1 /A/B/n1 +tree -o +log +index t33 +exit + diff --git a/contrib/fssim/testcases/daptest6.txt b/contrib/fssim/testcases/daptest6.txt new file mode 100644 index 00000000..bfb2f56f --- /dev/null +++ b/contrib/fssim/testcases/daptest6.txt @@ -0,0 +1,9 @@ +index q0 +touch a/b/c/foo +index q1 +touch a/b/c/foo +touch a/b/c +touch a/b +log +index q2 +exit diff --git a/contrib/fssim/testcases/fssimin-Log.txt b/contrib/fssim/testcases/fssimin-Log.txt new file mode 100644 index 00000000..9618232b --- /dev/null +++ b/contrib/fssim/testcases/fssimin-Log.txt @@ -0,0 +1,14 @@ +index l0 +touch A/D/bar +touch A/B/C/foo +echo "Hello world!" > A/B/C/foo +cd /A/B/C +cp foo / +cd +rm A/B/C/foo +rm foo +echo "foo!" > foo +log +log -t +index l1 +exit diff --git a/contrib/fssim/testcases/fssimin-TI.txt b/contrib/fssim/testcases/fssimin-TI.txt new file mode 100644 index 00000000..9998490f --- /dev/null +++ b/contrib/fssim/testcases/fssimin-TI.txt @@ -0,0 +1,77 @@ +index J0 +touch A1/B1/C1/c +echo "some " > A1/B1/b +md A1/B2 A1/B3 +tree +index J1 +#Case 1. Deleting a file +# step 1: delete /A1/B1/b +rm A1/B1/b +tree +index J2 +#Case 2. Adding data to a file, and rename the file +# step 1: adding data at end of existing file /A1/B1/b +# step 2: rename the file b to b.NEW +echo data >> A1/B1/b +mv A1/B1/b A1/B1/b.NEW +tree +index J3 +#Case 3: 1. Deleting a file (file UID: XXX), +# and create a new file with same name (file UID: YYY) +touch foo +ls +rm foo +touch foo +ls +index J4 +rm foo +#Case 4. Creating a new file with same name, after renaming the original one +# step 1: rename file b to b.bak +# step 2: create a new file as b +touch b +mv b b.bak +touch b +ls +index J5 +#Case 5. Moving whole subdirectory to another location +# step 1: move directory B1 under directory B2 +tree +mv A1/B1 A1/B2/B1 +tree +index J6 +#Case 6. Moving a file to other location, following by moving its parent +# directory to different location (or move a subdirectory first, and then +# move a file inside the subdirectory to other location). +# step 1: move file /A1/B1/C1/c to B2 (now it becomes /A1/B2/c) +# step 2: move directory B1 (not including c) under B3 +# or, +# step 1: move directory B1 under B3 +# step 2: move file c in /A1/B3/B1/C1 directory to B2 +md A1/B1/C1 A1/B3 +touch A1/B1/C1/c +tree +index XY +mv A1/B1/C1/c A1/B2 +mv A1/B1 A1/B3 +tree -o +log +index J7 +#Case 7. Moving a file more than once +# step 1: move file b to directory B2 +# step 2: move the same file (/A1/B2/b) to B3 +mv b /A1/B2 +ll +ls /A1/B2 +mv /A1/B2/b A1/B3 +ll /A1/B2 +ls /A1/B3 +tree +index J8 +#Case 8. Create a file and delete it within an interval +# step 1: delete file c +# step 2: create file c in /A1/B1/C +touch c +rm c +ll +index J9 +exit diff --git a/contrib/fssim/testcases/fssimin10.txt b/contrib/fssim/testcases/fssimin10.txt new file mode 100644 index 00000000..13902adc --- /dev/null +++ b/contrib/fssim/testcases/fssimin10.txt @@ -0,0 +1,63 @@ +index -f t10-0 +# /A +# |_ /B +# | |_ n1 +# |_ n2 +# |_ n3 +# /C +# |_ n4 +# |_ n5 +# /n1 +# /D +# /E +# +touch /A/B/n1 /A/n2 /A/n3 /C/n4 /C/n5 n1 +mkdir D E +tree +index -i t10-1 +fsck +# 1. Delete /A/B/n1 +# Rename the directory /C to /A/B/n1 +rm /A/B/n1 +mv /C /A/B/n1 +tree -o +index -i t10-2 +fsck +# 2. Append data data to /A/B/n1 +# Delete /A/B/n1 +# Rename file /n1 to /A/B/n1 +touch /A/B/n1 +rm /A/B/n1 +mv n1 /A/B/n1 +tree -o +index -i t10-3 +fsck +# 3. Delete /E +# Rename /D to /E +rd E +mv D E +tree +index -i t10-4 +fsck +# 4. Append data to /A/B/n1 +# Rename /A/B/n1 to /A/n1 +# Delete /A/B +# Rename /C to /A/B +# Move back /A/n1 to the new directory under /A/B +md C +touch /A/B/n1 +mv A/B/n1 A/n1 +rd A/B +mv C /A/B +mv n1 /A/B +tree +log +index t10-5 +fsck +# 5. Delete /n1 +# Create a new non-empty file /n1 +# Delete /n1 again +# Create an empty file /n1 + +exit + diff --git a/contrib/fssim/testcases/fssimin4.txt b/contrib/fssim/testcases/fssimin4.txt new file mode 100644 index 00000000..aa5cc5f5 --- /dev/null +++ b/contrib/fssim/testcases/fssimin4.txt @@ -0,0 +1,7 @@ +index t4-0 +touch asdf/foo +rm asdf/foo +rd asdf +index t4-1 +exit + diff --git a/contrib/fssim/testcases/fssimin5.txt b/contrib/fssim/testcases/fssimin5.txt new file mode 100644 index 00000000..036efda7 --- /dev/null +++ b/contrib/fssim/testcases/fssimin5.txt @@ -0,0 +1,8 @@ +index t5-0 +touch asdf/qwer/zxcv/foo +index t5-1 +rm asdf/qwer/zxcv/foo +ll asdf/qwer/zxcv +index t5-2 +ll asdf/qwer/zxcv +exit diff --git a/contrib/fssim/testcases/fssimin6.txt b/contrib/fssim/testcases/fssimin6.txt new file mode 100644 index 00000000..761dc985 --- /dev/null +++ b/contrib/fssim/testcases/fssimin6.txt @@ -0,0 +1,8 @@ +index t6-0 +md Dir1 +touch Dir2/Dir3/foo Dir2/Dir3/bar +index t6-1 +touch Dir2/Dir3/baz +rd Dir1 +index t6-2 +exit diff --git a/contrib/fssim/testcases/fssimin7.txt b/contrib/fssim/testcases/fssimin7.txt new file mode 100644 index 00000000..0638fba9 --- /dev/null +++ b/contrib/fssim/testcases/fssimin7.txt @@ -0,0 +1,35 @@ +index t7-0 +touch asdf/foo asdf/bar +rm asdf/foo +rm asdf/bar +# asdf has deleted files foo and bar +ll asdf +rd asdf +ll +# asdf is deleted +md asdf +ll +# asdf is recreated, has a different oid (so is different directory) +ll asdf +rd asdf +# asdf is deleted again +ll +touch qwer/baz qwer/bar +# qwer is created, contains only bar and baz +ll +ll qwer +tree -o +mv qwer asdf +# qwer (with bar and baz) is moved to deleted asdf, asdf has new oid +ll +# qwer is now deleted, asdf is recreated (share same oid) +ll asdf +# asdf should contain bar, baz +md qwer +# qwer is recreated, gets new oid so is different directory +ll +ll asdf +ll qwer +tree -o +index t7-1 +exit diff --git a/contrib/fssim/testcases/fssimin8.txt b/contrib/fssim/testcases/fssimin8.txt new file mode 100644 index 00000000..f438efdb --- /dev/null +++ b/contrib/fssim/testcases/fssimin8.txt @@ -0,0 +1,18 @@ +index t8-0 +touch asdf/foo asdf/bar +index t8-1 +# index t8-1 contains asdf with both foo and bar +rm asdf/foo +rm asdf/bar +rd asdf +# remove foo and bar, and directory asdf +touch asdf +#recreate asdf as a file +rm asdf +# now remove asdf +md asdf +# and recreate asdf as a directory again +index t8-2 +tree +exit + diff --git a/contrib/fssim/testcases/fssimin9.txt b/contrib/fssim/testcases/fssimin9.txt new file mode 100644 index 00000000..a55f2638 --- /dev/null +++ b/contrib/fssim/testcases/fssimin9.txt @@ -0,0 +1,17 @@ +index t9-0 +md qwer +touch asdf/zxcv/uiop/foo asdf/zxcv/uiop/bar asdf/zxcv/hjkl/baz +tree -o +# directory is qwer empty, adsf contains zxcv with subdirs and files +index t9-1 +mv asdf/zxcv qwer +tree -o +ll asdf +# move entire zxcv subtree to qwer, verify everything moves correctly +index t9-2 +md asdf/zxcv +cp -r qwer/zxcv asdf/zxcv +tree -o +index t9-3 +exit + diff --git a/contrib/fssim/testcases/logerr.txt b/contrib/fssim/testcases/logerr.txt new file mode 100644 index 00000000..df776b5d --- /dev/null +++ b/contrib/fssim/testcases/logerr.txt @@ -0,0 +1,19 @@ +touch A/B/foo +touch A/B/bar +touch A/B/baz +touch A/B/C/foo +touch A/B/C/bar +touch A/B/C/baz +log +index l0 +rm A/B/foo +rm A/B/bar +rm A/B/baz +rm A/B/C/foo +rm A/B/C/bar +rm A/B/C/baz +rd A/B/C +rd A/B +log +index l1 +exit diff --git a/contrib/fssim/testcases/tc001.txt b/contrib/fssim/testcases/tc001.txt new file mode 100644 index 00000000..2360faca --- /dev/null +++ b/contrib/fssim/testcases/tc001.txt @@ -0,0 +1,14 @@ +mkdir /DA1 +mkdir /DB1/DB2/DB3/DB4/DB5/DB6/DB7/DB8/DB9/DB10 +index initial_tc001 +mv /DB1/DB2/DB3/DB4/DB5/DB6/DB7/DB8/DB9/DB10 /DA1 +mv /DB1/DB2/DB3/DB4/DB5/DB6/DB7/DB8/DB9 /DA1/DB10 +mv /DB1/DB2/DB3/DB4/DB5/DB6/DB7/DB8 /DA1/DB10/DB9 +mv /DB1/DB2/DB3/DB4/DB5/DB6/DB7 /DA1/DB10/DB9/DB8 +mv /DB1/DB2/DB3/DB4/DB5/DB6 /DA1/DB10/DB9/DB8/DB7 +mv /DB1/DB2/DB3/DB4/DB5 /DA1/DB10/DB9/DB8/DB7/DB6 +mv /DB1/DB2/DB3/DB4 /DA1/DB10/DB9/DB8/DB7/DB6/DB5 +mv /DB1/DB2/DB3 /DA1/DB10/DB9/DB8/DB7/DB6/DB5/DB4 +mv /DB1/DB2 /DA1/DB10/DB9/DB8/DB7/DB6/DB5/DB4/DB3 +index flipped_tc001 +exit diff --git a/contrib/fssim/testcases/tc002.txt b/contrib/fssim/testcases/tc002.txt new file mode 100644 index 00000000..a901c792 --- /dev/null +++ b/contrib/fssim/testcases/tc002.txt @@ -0,0 +1,8 @@ +mkdir /DB0 +mkdir /DB1/DB2/DB3/DB4/DB5/DB6/DB7/DB8/DB9/DB10 +index initial_tc002 +mv /DB1/DB2 /DB0 +mv /DB0/DB2/DB3 /DB1 +mv /DB1/DB3/DB4 /DB0/DB2 +index shuffle_tc002 +exit diff --git a/contrib/fssim/testcases/tc003.txt b/contrib/fssim/testcases/tc003.txt new file mode 100644 index 00000000..e83016ee --- /dev/null +++ b/contrib/fssim/testcases/tc003.txt @@ -0,0 +1,29 @@ +mkdir /DB0 +mkdir /DB1/DB2/DB3/DB4/DB5/DB6/DB7/DB8/DB9/DB10 +tree +index initial_tc003 +mv /DB1/DB2 /DB0 +mv /DB0/DB2/DB3 /DB1 +mv /DB1/DB3/DB4 /DB0/DB2 +mv /DB0/DB2/DB4/DB5 /DB1/DB3 +mv /DB1/DB3/DB5/DB6 /DB0/DB2/DB4 +mv /DB0/DB2/DB4/DB6/DB7 /DB1/DB3/DB5 +mv /DB1/DB3/DB5/DB7/DB8 /DB0/DB2/DB4/DB6 +mv /DB0/DB2/DB4/DB6/DB8/DB9 /DB1/DB3/DB5/DB7 +mv /DB1/DB3/DB5/DB7/DB9/DB10 /DB0/DB2/DB4/DB6/DB8 +tree -o +mv /DB0 /DA1 +mv /DA1/DB2 /DA1/DA2 +mv /DA1/DA2/DB4 /DA1/DA2/DA3 +mv /DA1/DA2/DA3/DB6 /DA1/DA2/DA3/DA4 +mv /DA1/DA2/DA3/DA4/DB8 /DA1/DA2/DA3/DA4/DA5 +mv /DA1/DA2/DA3/DA4/DA5/DB10 /DA1/DA2/DA3/DA4/DA5/DA6 +mv /DB1/DB3/DB5/DB7/DB9 /DB1/DB3/DB5/DB7/DB1 +mv /DB1/DB3/DB5/DB7 /DB1/DB3/DB5/DB2 +mv /DB1/DB3/DB5 /DB1/DB3/DB3 +mv /DB1/DB3 /DB1/DB4 +mv /DB1 /DB5 +tree -o +#log +index shuffle_tc003 +exit diff --git a/contrib/fssim/testcases/tc004.txt b/contrib/fssim/testcases/tc004.txt new file mode 100644 index 00000000..de81eaab --- /dev/null +++ b/contrib/fssim/testcases/tc004.txt @@ -0,0 +1,34 @@ +mkdir /DB0 +mkdir /DB1/DB2/DB3/DB4/DB5/DB6/DB7/DB8/DB9/DB10 +index initial_tc004 +mv /DB1/DB2 /DB0 +mv /DB0/DB2/DB3 /DB1 +mv /DB1/DB3/DB4 /DB0/DB2 +mv /DB0/DB2/DB4/DB5 /DB1/DB3 +mv /DB1/DB3/DB5/DB6 /DB0/DB2/DB4 +mv /DB0/DB2/DB4/DB6/DB7 /DB1/DB3/DB5 +mv /DB1/DB3/DB5/DB7/DB8 /DB0/DB2/DB4/DB6 +mv /DB0/DB2/DB4/DB6/DB8/DB9 /DB1/DB3/DB5/DB7 +mv /DB1/DB3/DB5/DB7/DB9/DB10 /DB0/DB2/DB4/DB6/DB8 +mv /DB0 /DA1 +mv /DA1/DB2 /DA1/DA2 +mv /DA1/DA2/DB4 /DA1/DA2/DA3 +mv /DA1/DA2/DA3/DB6 /DA1/DA2/DA3/DA4 +mv /DA1/DA2/DA3/DA4/DB8 /DA1/DA2/DA3/DA4/DA5 +mv /DA1/DA2/DA3/DA4/DA5/DB10 /DA1/DA2/DA3/DA4/DA5/DA6 +mv /DB1/DB3/DB5/DB7/DB9 /DB1/DB3/DB5/DB7/DB1 +mv /DB1/DB3/DB5/DB7 /DB1/DB3/DB5/DB2 +mv /DB1/DB3/DB5 /DB1/DB3/DB3 +mv /DB1/DB3 /DB1/DB4 +mv /DB1 /DB5 +tree +rmdir /DA1/DA2/DA3/DA4/DA5/DA6 +rmdir /DA1/DA2/DA3/DA4/DA5 +rmdir /DA1/DA2/DA3/DA4 +rmdir /DA1/DA2/DA3 +rmdir /DB5/DB4/DB3/DB2/DB1 +rmdir /DB5/DB4/DB3/DB2 +rmdir /DB5/DB4/DB3 +tree +index shuffle_remove_tc004 +exit diff --git a/contrib/fssim/testcases/tc005.txt b/contrib/fssim/testcases/tc005.txt new file mode 100644 index 00000000..04620dad --- /dev/null +++ b/contrib/fssim/testcases/tc005.txt @@ -0,0 +1,13 @@ +mkdir /DA0 +mkdir /DB0 +touch /DA0/fa0 +touch /DB0/fb0 +tree +index initial_tc005 +rm /DA0/fa0 +touch /DA0/fa0 +mv /DB0/fb0 /DB0/fb1 +mv /DB0/fb1 /DB0/fb0 +tree +index new_samename_tc005 +exit From 13f43ef96f799b11e12bc00bca45a5b1f2c9ff94 Mon Sep 17 00:00:00 2001 From: DavidAPease Date: Thu, 1 Oct 2020 13:34:27 -0700 Subject: [PATCH 003/121] Corrected typos in README files --- contrib/fssim/README.md | 2 +- contrib/fssim/docs/README.txt | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/contrib/fssim/README.md b/contrib/fssim/README.md index d9200886..aeb53b36 100644 --- a/contrib/fssim/README.md +++ b/contrib/fssim/README.md @@ -1,6 +1,6 @@ # LTFS File System Simulator (fssim) -The major change in version 2.5 of the [LTFS Format Specification](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) is the addition of Incremental Indexes. Incremental Indexes allow an LTFS index written to the Data Partition of the tape to contain only the changes from the prior index, rather than the full state of the file system. For a full discussion of Incremental Indexes, see Appendix H of the Format Specification reference above. +The major change in version 2.5 of the [LTFS Format Specification](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) is the addition of Incremental Indexes. Incremental Indexes allow an LTFS index written to the Data Partition of the tape to contain only the changes from the prior index, rather than the full state of the file system. For a full discussion of Incremental Indexes, see Appendix H of the Format Specification referenced above. When we first proposed Incremental Indexes there was considerable discussion concerning what they would need to contain, and how they would be implemented. I decided to write a simulator that would mimic file system operations, and create full and incremental XML indexes very similar to those in LTFS as a Proof of Concept. The simulator package also includes programs to merge such directories and verify the result, and a directory of test cases built by various members of the LTFS project. diff --git a/contrib/fssim/docs/README.txt b/contrib/fssim/docs/README.txt index 90682776..d9966735 100644 --- a/contrib/fssim/docs/README.txt +++ b/contrib/fssim/docs/README.txt @@ -19,7 +19,9 @@ sequentially in the same execution of fssim, then creates a full index from the result. The resulting full index should exactly match the corresponding full index from fssim. -The programs were written by David Pease at IBM's Almaden Research Center in May-August of 2018, and significantly revised and improved by the original author (now retired) in September of 2020. +The programs were written by David Pease at IBM's Almaden Research Center in +May-August of 2018, and significantly revised and improved by the original +author (now retired) in September of 2020. Invoking the programs: ====================== From 5b6b5e449b632774aae5d1fb45ae016be158e8f7 Mon Sep 17 00:00:00 2001 From: DavidAPease Date: Thu, 1 Oct 2020 13:45:44 -0700 Subject: [PATCH 004/121] Updates to README files --- contrib/fssim/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/contrib/fssim/README.md b/contrib/fssim/README.md index aeb53b36..c4550c18 100644 --- a/contrib/fssim/README.md +++ b/contrib/fssim/README.md @@ -2,7 +2,9 @@ The major change in version 2.5 of the [LTFS Format Specification](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) is the addition of Incremental Indexes. Incremental Indexes allow an LTFS index written to the Data Partition of the tape to contain only the changes from the prior index, rather than the full state of the file system. For a full discussion of Incremental Indexes, see Appendix H of the Format Specification referenced above. -When we first proposed Incremental Indexes there was considerable discussion concerning what they would need to contain, and how they would be implemented. I decided to write a simulator that would mimic file system operations, and create full and incremental XML indexes very similar to those in LTFS as a Proof of Concept. The simulator package also includes programs to merge such directories and verify the result, and a directory of test cases built by various members of the LTFS project. +When we first proposed Incremental Indexes there was considerable discussion concerning what they would need to contain, and how they would be implemented. I decided to write a simulator that would mimic file system operations, and create full and incremental XML indexes very similar to those in LTFS as a Proof of Concept. This simulator also includes programs to merge full and incremental directories created by it and verify the results, as well as a directory of test cases built by various members of the LTFS project. + +The programs are meant to demonstrate how Incremental Indexes work, and also to potentially serve as an example of how they could be implemented. Full documentation of the programs is contained in the README file in the docs directory. From 076ff6d88ff06a0565e420accb0f40a71ede97ec Mon Sep 17 00:00:00 2001 From: DavidAPease Date: Sat, 3 Oct 2020 19:52:07 -0700 Subject: [PATCH 005/121] README.md in docs dir, small fixes to help messages --- contrib/fssim/docs/README.md | 381 ++++++++++++++++++++++++++++++++++ contrib/fssim/docs/README.txt | 2 +- contrib/fssim/src/fssim | 2 +- 3 files changed, 383 insertions(+), 2 deletions(-) create mode 100644 contrib/fssim/docs/README.md diff --git a/contrib/fssim/docs/README.md b/contrib/fssim/docs/README.md new file mode 100644 index 00000000..085c0b92 --- /dev/null +++ b/contrib/fssim/docs/README.md @@ -0,0 +1,381 @@ + +# LTFS File System Simulator (fssim) +# Usage Guide and Examples + + +## Overview + +The File System Simulator programs are used to simulate file system activity, +create full and incrmental indexes reflecting that activity, update a full +index using incremental indexes, and verify the correctness of that update. + +The first program in the set is _fssim_. It simulates file system activity such +as creating or deleting files and directories, moving and copying files or +directories, listing information, etc. It also writes full and incremental +XML indexes to text files on request. + +The second program is _fsidxupd_ (perhaps a poor choice of names). It updates +a full index created by fssim with one or more incremental indexes created +sequentially in the same execution of fssim, then creates a full index +from the result. The resulting full index should exactly match the +corresponding full index from fssim. + +The programs were written by David Pease at IBM's Almaden Research Center in +May-August of 2018, and significantly revised and improved by the original +author (now retired) in September of 2020. + +## Invoking the programs: + +The programs are python executables, meaning that they can be invoked from +the command line in Unix-like environments (such as Linux and Mac) as long as +the executable bit is on and /usr/bin/python points to the python interpreter. The programs are written so that they run correctly in either python 2 or +python 3. + +In Windows, it should be possible to invoke them by a command such as "python +fsXXX" or by renaming the two program files to fsXXX.py and invoking them by +name (if python is properly configured). + +There are two additional files besides the _fssim_ and _fsidxupd_ programs. They +are _fscommon.py_ and _fsglobals.py_. Both of these are imported by the two +command programs; all four files should reside in the same directory. + +### fssim + +The format of the _fssim_ command is: + + fssim [-d] [command file] + +where: +* -d: prints a ton of debugging messages (probably meaningfull only to me) during processing +* command file: executes commands from the specified text file at startup + + A command file is simply a text file containing commands as they would be + entered from the keyboard; blank lines and comment lines starting with "#" + are allowed. Command files make it easy to design and debug a scenario, + then execute it multiple times (often while modifying code + in-between). After a command file is executed (without encountering an + "exit" command), the program drops into its normal keyboard input mode + (which is terminated by entering "exit"). + +### fsidxupd + +The format of the _fsidxupd_ command is: + + fsidxupd [-d] [ ...] + +where: +* -d: prints lots of debugging messages during processing +* fullindex: a file containing a full index from fssim +* incrindexes:a list of files containing incremental indexes from fssim + +The first file name must identify a full index created by _fssim_. The second +and subsequent names, if present, must identify incremental indexes created +sequentially by _fssim_. The output of the program (currently) has a hard-coded +file name of _upd-full.xml_. + +## fssim Commands + +The _fssim_ commands are closely modeled after the Unix shell commands that +perform the same function; however, they are often greatly simplified versions +(the goal was to implement a simulator, not re-write bash or busybox). + +Unrecognized commands or commands with an invalid number of operands will +return an **"\*\* Invalid"** message. Commands that do not execute successfully +(for any reason) will return the message **"\*\* Failed"**. That is the extent +of error messages. + +The command prompt indicates the current working directory (as it typically +does in bash), and starts at root ("/"). File and directory names can be +entered as full paths or relative to the current directory; however, the only +place that ".." is recognized is as the single operand of a _cd_ command, and +the only place that "." is recognized is as the single target for a _mv_ or _cp_ command. The directory separator character is "/". The command prompt +environment supports command recall and editing using the _readline_ library (tested on Linux and Mac). + +The command processor respects strings enclosed in double quotes ("). Quoted +strings can be used to create object names with embedded blanks or +to echo strings with arbitrary blank spaces. For example: + + echo Hello > "file 1"` + echo " world!" >> "file 1"` + cat "file 1"` + +The _help_ command will print a short description of all of the commands. +Here is a slightly more detailed command summary (square brackets indicate +optional parameters, curly braces indicate a choice): + +|name|operands|description| +|:-|:-|:-| +|cat|name|write a file's contents to the screen| +|cd|[dirname]|change current directory; with no operands, changes to root| +|cls||clear the screen (tested in Linux terminal window)| +|cp|[-r] name name|copy a file from one name to another; with -r option, copy directory contents recursively| +|dir|[-a] [name]|an alias for ls| +|echo|data [{>\|>>} name]|write data to the screen or to a file| +|fsck||verify that all object flags have been correctly reset | +|index|[{-i\|-f}] fileprefix|write full and/or incremental indexes to files using names prefixed with the specified prefix (see below)| +|ll|[name]|list long: an alias for ls -a | +|log|[-t]|list current update log entries (optionally in time order)| +|ls|[-a] [name]|list file or directory info (see more info below)| +|md|dirname [...]|an alias for mkdir| +|mkdir|dirname[...]|create directories or full directory paths| +|mv|name name|move/rename a file or directory| +|rd|dirname|an alias for rmdir| +|rm|filename|remove a file| +|rmdir|dirname|remove a directory (if it is empty)| +|touch|name [...] |update the timestamp of existing files or directories, or create new files (with directory paths if needed)| +|tree|[-o]|show a tree view of the entire file system; with -o option, also include oids| + +The _ls_ (or _dir_) command prefixes each file or directory with some attributes in +the order **"dmn"**. A **"d"** in the first position indicates a directory. An **"m"** +in the second position indicates that the file or directory has been modified +since the last index was written. An **"n"** in the third position indicates that +the directory is new since the last index (and thus must be written in its entirety with all of its children to an incremental index). Following the +attributes are the object id, the modification time, and the object size. For +files, the size is the number of data bytes in the file; for directories, it +is the number of children in the directory. When the object of the _ls_ +command is a directory, the "-a" (all) option will also show an entry for "." +which lists the status of the directory itself. (The _ll_ command is shorthand +for _ls -a_.) + +The _tree_ command shows a full file system tree view; using the "-o" option +shows the object ids (oids) following the object names, as in this example: + + / (0) + |-file1 (1) + |=dir1 (2) + |=dir2 (4) + |-file2 (5) + |=dir3 (3) + +As can be seen from the example, a "-" in front of a name in the tree view +indicates a file, while "=" indicates a directory. Within a directory +files are listed first followed by subdirectories, in alphabetical order. + +The _mkdir_ (or _md_), _touch_, and _echo_ commands will create any missing lower-level +directories required in the directory or file path. + +A note using on the _cp_ command with the "-r" option (recursive copy). This +command option requires two existing directory names as input; it copies the +**contents of the first directory to the second, recursively. Thus, using a +directory structure such as that shown in the output of _tree_ above, the +following command: + + cp -r dir1/dir2 dir3 + +would copy only file2 to dir3. In order to end up with dir2 and file2 under +dir3, one could use either + + cp -r dir1 dir3 + +or + + md dir3/dir2 + cp -r dir1/dir2 dir3/dir2 + +Note, however, that the first option will copy everything under dir1 to dir3; +if dir1 had files or directories other than dir2 under it, it would copy +everything and would not give the desired results. + +## Indexes + +By default, the _index_ command writes both a full index and an increment index +to a pair of files whose names start with the specified prefix. For example, +the command _index t0_ will create files _t0-full.xml_ and _t0-incr.xml_ in +the user's current directory. (The rationale for writing both indices is that +that one can start from any full index written, apply any number of subsequent +incremental indexes they wish, and the result should be the same as the full +index corresponding to the last incremental index used.) The _index_ command +allow overriding the default behavior by using the "-i" (incremental index +only) or "-f" (full index only) options. To simulate normal system operation, +(and minimize the number of files produced during testing), one could specify +"-f" for the index produced at the start of the run (e.g., _index -f t0_), then +use "-i" for all subsequent indexes up to the last one, and use neither flag +for the final index. This would give a starting and ending full index, and a +full set of incremental indexes to validate against the final full index. + +When testing indexes it's probably a good idea to start a command file with +the command _index [-f] t0_ (or the prefix of your choice); this creates a +starting point for incremental indexes from the beginning of the session. _index_ commands can be interspersed in a command stream at any point, and the +resulting full and incremental indexes can be inspected (and processed using _fsidxupd_). + +The _fsck_ command searches the entire file system for any files or directories +with **New** or **Modified** flags. This command can be used after an _index_ command with the +"-i" (incremental only) flag to verify that all objects with these flags have +been visited, and that all such flags have been correctly reset. + +A pair of "t0" indexes (created at startup) looks like this: + + + 0 + + + + + + + 0 + + + + + +The two are essentially identical, since the root directory is the only one in +existence, and it is new and thus shows up in the incremental index as well. (Actually, the t0 incremental index is useless, but it helps illustrate how +indexes work.) + +Running _fssim_ with this set of commands (e.g., in a command file): + + index t0 + mkdir Dir1 + touch Dir2/Dir3/foo + touch Dir2/Dir3/bar + index t1 + touch Dir2/Dir3/baz + rmdir Dir1 + index t2 + +produced the following two "t2" indexes: + + + 0 + + + + Dir2 + 2 + + + + Dir3 + 3 + + + + 5 + bar + + + + 6 + baz + + + + 4 + foo + + + + + + + + + + + + + + Dir1 + + + + Dir2 + + + Dir3 + + + + baz + 6 + + + + + + + + + +## Using fsidxupd + +_fsidxupd_ is used to verify the correctness of incremental indexes and their +application to a full index. For example, using the sample list of commands +in the Index section above, three pairs of indexes will be created by fssim +when running the commands; the file names will be _t0-full.xml, t0-incr.xml, +t1-full.xml, t1-incr.xml, t2-full.xml_, and _t2-incr.xml_. The following +_fsidxupd_ command will take the starting full index and apply the two appropriate incremental indexes: + + fsidxupd t0-full.xml t1-incr.xml t2-incr.xml + +The output of this command will be a file named _upd-full.xml_. In this example, +if the system is working correctly _upd-full.xml_ will be identical to +_t2-full.xml_. In Unix-like systems, the command: + + diff upd-full.xml t2-full.xml + +would show no output (no differences). + +## Program Organization + +The file _fscommon.py_ contains all of the file system logic. It contains +object definitions for directory and file objects, and implements all of +the low-level logic for managing those objects. (In function, it roughly +corresponds to the vfs module in a Unix-like OS.) It also contains the +the python version-agnostic print function (_fsprt_). + +The file _fsglobals.py_ defines global variables that must be shared between +the different modules. In order to provide a consistent naming reference +to these shared global variables, they must be in a separate python module +and that module must be imported with the same name in each program. (For +succinctness, I import it as "g", so that global values are referenced as +_g.variablename_.) + +The file _fssim_ is the command processor for user file system commands; it +processes commands from a command file and/or the command line. (In Unix +terms, it implements the user shell and shell commands.) + +The file _fsidxupd_ is a utility program that reads, updates, and writes +indexes. It uses file system functions from _fscommon.py_ to update a full +index using data from incremental indexes. + +## fsruntest + +The programs also include a bash shell script to automate testing. The +format of the command is: + + fsruntest [-d] + +The command file is used to run the test. It may end with an _exit_ command, +or the _exit_ command can be entered manually while the test is running. The +"-d" option runs _fssim_ and _fsidxupd_ with debugging messages enabled. + +The script first invokes _fssim_, passing in the name of the command file. It +then invokes _fsidxupd_ to apply all of the subsequent incremental indexes to +the first full index file created. Finally, it compares the output of _fsidxupd_ +to the final full index created by _fssim_. If the output of diff is not empty, +the output is displayed, preceded by a highlighted error line (so that it stands out or +can be searched for in a large test run). + +Since it is a bash shell script, the test script runs on both Linux and Mac +systems, and with the proper Linux subsystem configured perhaps even on +Windows 10 (or earlier, using Cygwin). + + +## Final Comments + +The _oid_s in the files and directories correspond to the _fileuid_ in LTFS. +Initially I did not believe that _oid_s (and thus _fileuid_s) were important +to processing incremental indexes. It turns out that I was wrong, and that +a unique identifier for an object is important to the correct application +of an incremental index in one important case. Timestamps are the last +object modification time. I think all other object fields are obvious. + +The programs have not been intensively debugged. There may be lingering +errors in the commands and/or in index generation. It all seems to work, +but if you find errors (or even think you **may** have found an error) please +let me know. The better debugged the programs are, the more reliable our +results will be. + +#### - David Pease - pease@coati.com diff --git a/contrib/fssim/docs/README.txt b/contrib/fssim/docs/README.txt index d9966735..f1918670 100644 --- a/contrib/fssim/docs/README.txt +++ b/contrib/fssim/docs/README.txt @@ -110,7 +110,7 @@ cls - clear the screen (tested in Linux terminal window) cp [-r] name name - copy a file from one name to another; with -r option, copy directory contents recursively dir [-a] [name] - an alias for ls -echo [data] [{>|>>} name] - write data to the screen or to a file +echo data [{>|>>} name] - write data to the screen or to a file fsck - verify that all object flags have been correctly reset index [{-i|-f}] fileprefix - write full and/or incremental indexes to files using names prefixed with the specified prefix (see below) diff --git a/contrib/fssim/src/fssim b/contrib/fssim/src/fssim index ac4ff394..be368613 100755 --- a/contrib/fssim/src/fssim +++ b/contrib/fssim/src/fssim @@ -326,7 +326,7 @@ cmdTable = { "dir": (0, 1, ls, "[-a] [name] - alias for ls"), "ll": (0, 1, ll, "[name] - alias for ls -a (ls with all info)"), "tree": (0, 0, tr, "[-o] - show a tree view of the file system"), - "echo": (1, 20,ec, "[data] [>|>> name] - write data to screen or file"), + "echo": (1, 20,ec, "data [>|>> name] - write data to screen or file"), "cat": (1, 1, ct, "name - write a file's contents to the screen"), "cls": (0, 0, cl, " - clear the screen"), "index": (1, 1, ix, "[-f|-i] fileprefix - write full and/or incr indexes"), From 3dc47e1283f8d23550b6da2921e7df3b65d091ca Mon Sep 17 00:00:00 2001 From: DavidAPease Date: Sat, 3 Oct 2020 19:54:51 -0700 Subject: [PATCH 006/121] typos in README --- contrib/fssim/docs/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/contrib/fssim/docs/README.md b/contrib/fssim/docs/README.md index 085c0b92..cb1a2fac 100644 --- a/contrib/fssim/docs/README.md +++ b/contrib/fssim/docs/README.md @@ -365,8 +365,8 @@ Windows 10 (or earlier, using Cygwin). ## Final Comments -The _oid_s in the files and directories correspond to the _fileuid_ in LTFS. -Initially I did not believe that _oid_s (and thus _fileuid_s) were important +The _oidsi_ in the files and directories correspond to the _fileuid_ in LTFS. +Initially I did not believe that _oids_ (and thus _fileuids_) were important to processing incremental indexes. It turns out that I was wrong, and that a unique identifier for an object is important to the correct application of an incremental index in one important case. Timestamps are the last From d35cd787f2602de56046d170bb8aa6073adb8273 Mon Sep 17 00:00:00 2001 From: DavidAPease Date: Sat, 3 Oct 2020 19:55:39 -0700 Subject: [PATCH 007/121] typos in README --- contrib/fssim/docs/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/fssim/docs/README.md b/contrib/fssim/docs/README.md index cb1a2fac..06300c54 100644 --- a/contrib/fssim/docs/README.md +++ b/contrib/fssim/docs/README.md @@ -365,7 +365,7 @@ Windows 10 (or earlier, using Cygwin). ## Final Comments -The _oidsi_ in the files and directories correspond to the _fileuid_ in LTFS. +The _oids_ in the files and directories correspond to the _fileuid_ in LTFS. Initially I did not believe that _oids_ (and thus _fileuids_) were important to processing incremental indexes. It turns out that I was wrong, and that a unique identifier for an object is important to the correct application From b846d2763ca3a0e12f4072a49cdeeebd4e1830e7 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 6 Nov 2020 17:49:08 +0900 Subject: [PATCH 008/121] Make master branch as a tree for version 2.5 dev --- README.md | 4 ++++ configure.ac | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 21d978c7..640f4df3 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,10 @@ [![CodeFactor](https://www.codefactor.io/repository/github/lineartapefilesystem/ltfs/badge)](https://www.codefactor.io/repository/github/lineartapefilesystem/ltfs) [![BSD License](http://img.shields.io/badge/license-BSD-blue.svg?style=flat)](LICENSE) +# About this branch + +This is the `master` branch of the LTFS project. At this time, this branch is used for version 2.5 development. So it wouldn't be stable a little. Please consider to follow the tree on `v2.4-stable` branch if you want to use stable codes. + # Linear Tape File System (LTFS) Linear Tape File System (LTFS) is a filesystem to mount a LTFS formatted tape in a tape drive. Once LTFS mounts a LTFS formatted tape as filesystem, user can access to the tape via filesystem API. diff --git a/configure.ac b/configure.ac index 8ad4062d..b8a2fee0 100644 --- a/configure.ac +++ b/configure.ac @@ -36,7 +36,7 @@ dnl dnl LTFS configure.ac. dnl -AC_INIT([LTFS], [2.4.3.1 (Prelim)], IBM corporation.) +AC_INIT([LTFS], [2.5.0.0 (Prelim)], IBM corporation.) AC_CONFIG_SRCDIR([src/main.c]) AC_CONFIG_AUX_DIR([build-aux]) AC_CONFIG_MACRO_DIRS([m4]) From 26869c80ea0bd0c69b91c6f081b4a97cd45860b7 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 13 Nov 2020 15:29:27 +0900 Subject: [PATCH 009/121] Update build-centos7.yml --- .github/workflows/build-centos7.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build-centos7.yml b/.github/workflows/build-centos7.yml index f00d5e53..6fd8ddbd 100644 --- a/.github/workflows/build-centos7.yml +++ b/.github/workflows/build-centos7.yml @@ -12,6 +12,6 @@ jobs: uses: actions/checkout@v1 - name: Build LTFS id: build - uses: LinearTapeFileSystem/CentOS7-Build@v1.2 + uses: LinearTapeFileSystem/CentOS7-Build@v1.3 with: destination: '/tmp/ltfs' From 2e117ad691b6a34f70b28d114abb5e951dff6c72 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 13 Nov 2020 15:31:43 +0900 Subject: [PATCH 010/121] Fix warning on message checker --- src/libltfs/ltfs.c | 1 - src/ltfsmsg.h | 1502 ++++++++++++++++++++++ src/tape_drivers/linux/sg/quantum_tape.c | 1 + src/tape_drivers/linux/sg/sg_tape.c | 4 +- 4 files changed, 1505 insertions(+), 3 deletions(-) create mode 100644 src/ltfsmsg.h create mode 120000 src/tape_drivers/linux/sg/quantum_tape.c diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index fa6a358c..10a1bd39 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1886,7 +1886,6 @@ int ltfs_unmount(char *reason, struct ltfs_volume *vol) int ret; cartridge_health_info h; int vollock = UNLOCKED_MAM; - char *skip_reason = NULL; char *mount_type = NULL; char *mam_lock = NULL; diff --git a/src/ltfsmsg.h b/src/ltfsmsg.h new file mode 100644 index 00000000..06725ac6 --- /dev/null +++ b/src/ltfsmsg.h @@ -0,0 +1,1502 @@ +#define LTFS16420I " -v, --traverse= Set traverse mode for listing roll back points. Strategy should be forward or backward. (default: backward)" +#define LTFS16088I "Launched by \"%s\"." +#define LTFS16028I "Compression : %s." +#define LTFS16081D "Latest index generation is %d (%c, %llu)." +#define LTFS16004E "Unexpected condition: str_gen is not specified." +#define LTFS16104E "Could not initialize the key manager interface plug-in. \'%s\' (%d)." +#define LTFS16080E "Cannot check volume (%d)." +#define LTFS16096W "Both EODs are missing. Attempt to list index information." +#define LTFS16020W "Failed to close the device (%d)." +#define LTFS16032I "Size threshold : %llu." +#define LTFS16068E "Cannot roll back: found 2 or more target indexes in one partition (%d)." +#define LTFS16052D "Erase history: spacing back and writing a file mark." +#define LTFS16057E "Cannot roll back: the medium is read-only." +#define LTFS16413I " -p, --advanced-help Full help, including advanced options" +#define LTFS16404I " -r, --rollback Roll back to the point specified by -g" +#define LTFS16403I " -g, --generation= Specify the generation to roll back" +#define LTFS16421I " -z, --deep-recovery Recover EOD missing cartridge.\n" +#define LTFS16089I "%s." +#define LTFS16110E "The --salvage-rollback-points option was specified against a normal cartridge." +#define LTFS16029I "Index partition : ID = %c, SCSI Partition = %u." +#define LTFS16005E "Invalid generation number %s." +#define LTFS16024I "Volser (bar code) : %s." +#define LTFS16107E "Unknown option '%s %s'." +#define LTFS16076E "Cannot list rollback points: failed to traverse the data partition (%d)." +#define LTFS16011E "Cannot open device \'%s\'." +#define LTFS16060E "Cannot roll back the index partition: failed to write an index (%d)." +#define LTFS16056E "Cannot roll back the data partition: failed to write an index (%d)." +#define LTFS16025I "Volume UUID : %s." +#define LTFS16019E "Invalid operation mode." +#define LTFS16021E "Volume is inconsistent and was not corrected." +#define LTFS16412I " -h, --help This help" +#define LTFS16059E "Cannot roll back the index partition: failed to erase history (%d)." +#define LTFS16086I "Volume is rolled back successfully." +#define LTFS16062I "Roll back from the data partition." +#define LTFS16051E "Cannot erase history: failed to space forward 1 file mark (%d)." +#define LTFS16090I "GCC version is %s." +#define LTFS16002E "Option validation failed." +#define LTFS16422I " -m, --full-index-info Display full index information (Effective only for -l option)" +#define LTFS16014I "Checking LTFS file system on \'%s\'." +#define LTFS16030I "Data partition : ID = %c, SCSI Partition = %u." +#define LTFS16075E "Cannot list rollback points: failed to traverse the index partition (%d)." +#define LTFS16102E "Cannot open key manager interface backend \'%s\'." +#define LTFS16045D "Erasing history from (%c, %llu)." +#define LTFS16415I " -e, --backend= Override the default tape device backend" +#define LTFS16022I "Volume is consistent." +#define LTFS16408I " -j, --erase-history Erase history at rollback" +#define LTFS16401I "filesys Device file for the tape drive" +#define LTFS16099E "Use the latest version of LTFS software." +#define LTFS16423I " --kmi-backend= Override the default key manager interface backend" +#define LTFS16400I "Usage: %s [options] filesys" +#define LTFS16087E "Volume is inconsistent. Try to recover consistency with ltfsck first." +#define LTFS16058I "Rolling back from the index partition." +#define LTFS16097E "Both EODs are missing. Roll back operation is not permitted." +#define LTFS16063I "Specified rollback point is the current index. The volume is consistent. No operation is required." +#define LTFS16050D "Erase history: spacing to end of index." +#define LTFS16074E "Cannot list rollback points: failed to load the volume (%d)." +#define LTFS16003E "Must provide search criteria using -g." +#define LTFS16105E "Key manager interface backend option parsing failed." +#define LTFS16033I "Name pattern : %s." +#define LTFS16046D "Rolling back %s: (%c, %llu)." +#define LTFS16414I " -i, --config= Use the specified configuration file (default: %s)" +#define LTFS16023I "LTFS volume information:." +#define LTFS16402I "Available options are:" +#define LTFS16098E "Cannot roll back the cartridge: found unsupported index version." +#define LTFS16109E "This operation is not allowed on this medium (%s)." +#define LTFS16416I " -x, --fulltrace Enable full function call tracing (slow)" +#define LTFS16016E "Invalid search mode." +#define LTFS16424I " --capture-index Capture index information to the current directory (-g is effective for this option)" +#define LTFS16092E "Cannot set up tape drive (%s)." +#define LTFS16108I "%s version %s." +#define LTFS16084I "List indexes in backward direction strategy." +#define LTFS16053E "Cannot erase history: failed to space back 1 file mark (%d)." +#define LTFS16073E "Cannot roll back: failed to find indexes." +#define LTFS16100E "Cannot recover the cartridge: found unsupported index version." +#define LTFS16000I "Starting ltfsck, %s version %s, log level %d." +#define LTFS16407I " -l, --list-rollback-points List rollback points" +#define LTFS16409I " -k, --keep-history Keep history at rollback (default)" +#define LTFS16085E "Unexpected traverse strategy." +#define LTFS16091E "Cannot recover missing EOD (%d)." +#define LTFS16425I " --syslogtrace Enable diagnostic output to stderr and syslog" +#define LTFS16015I "Rolling back LTFS file system on \'%s\'." +#define LTFS16001E "Cannot allocate LTFS volume structure." +#define LTFS16072E "Cannot roll back: failed to traverse the data partition (%d)." +#define LTFS16103W "Cannot unload key manager interface backend." +#define LTFS16009E "Must provide device name." +#define LTFS16031I "Data placement policy information:." +#define LTFS16094E "CM in the cartridge might be corrupted. Try to run ltfs with the \"-o force_mount_no_eod\" option." +#define LTFS16426I " -V, --version Version information" +#define LTFS16079E "Cannot roll back: failed to save index partition append position (%d)." +#define LTFS16082I "Saving latest index to data partition to save history." +#define LTFS16106E "Tape backend option parsing failed." +#define LTFS16006I "Rolling back to generation %d." +#define LTFS16010E "Cannot load backend \'%s\' (%d)." +#define LTFS16055E "Cannot roll back the data partition: failed to erase history (%d)." +#define LTFS16071E "Cannot roll back: failed to traverse the index partition (%d)." +#define LTFS16034I "Policy update : %s." +#define LTFS16018I "Listing LTFS file system rollback points on \'%s\'." +#define LTFS16411I " -t, --trace Enable diagnostic output" +#define LTFS16026I "Format time : %04d-%02d-%02d %02d:%02d:%02d.%09ld %s." +#define LTFS16405I " -n, --no-rollback Do not roll back. Verify the point specified by -g (default)" +#define LTFS16427I " --salvage-rollback-points List the rollback points of the cartridge that has no EOD" +#define LTFS16017I "Verify rollback point on \'%s\'." +#define LTFS16083I "List indexes in forward direction strategy." +#define LTFS16093E "Cannot recover the cartridge with ltfsck." +#define LTFS16027I "Block size : %lu." +#define LTFS16054E "Cannot erase history: failed to write a file mark (%d)." +#define LTFS16070E "Cannot roll back: failed to load the volume (%d)." +#define LTFS16111I "The recovery process is skipped because of a locked cartridge (%d)." +#define LTFS16067I "Rolling back based on the following index chain." +#define LTFS16101E "Please use the latest version os LTFS software or --deep-recovery option." +#define LTFS16406I " -f, --full-recovery Recover extra data blocks into directory %s" +#define LTFS16061E "Cannot roll back: invalid partition ID %c." +#define LTFS16410I " -q, --quiet Suppress informational messages" +#define LTFS31008E "Cannot locate: invalid partition %lu." +#define LTFS31018E "Cannot lock medium: unit not ready." +#define LTFS31026I "Getting the device directory (%s)." +#define LTFS31198D "Backend %s: (%llu, %llu) FM = %llu." +#define LTFS31028I "Scanning the devices directory (%s)." +#define LTFS31021E "Cannot erase: unit not ready." +#define LTFS31000I "Opening a device through generic itdtimage driver (%s)." +#define LTFS31012E "Cannot read position: unit not ready." +#define LTFS31002E "Failed to seek to %lld (%s, %lld)." +#define LTFS31010E "Cannot space: Unrecognized space type." +#define LTFS31016E "Cannot get remaining capacity: unit not ready." +#define LTFS31004D "Backend read: %llu bytes (from position=(%u, %llu), FMs %llu)." +#define LTFS31020D "Read attribute: %d %x." +#define LTFS31006E "Cannot rewind: unit not ready." +#define LTFS31014E "Cannot format: must issue command from partition 0, block 0." +#define LTFS31024E "Cannot set compression: unit not ready." +#define LTFS31197D "Backend %s: (%llu, %llu)." +#define LTFS31195D "Backend %s." +#define LTFS31009E "Cannot space: unit not ready." +#define LTFS31019E "Cannot unlock medium: unit not ready." +#define LTFS31007E "Cannot locate: unit not ready." +#define LTFS31030D "Found a device (%s, %s, %s, %s)." +#define LTFS31199I "itdtimg backend options:\n -o devname= tape device (default=%s)\n." +#define LTFS31027I "No device directory is specified (%s)." +#define LTFS31029E "Cannot open the device directory (%s)." +#define LTFS31001E "Failed to open %s: %s (%llu)." +#define LTFS31013E "Cannot set capacity: must issue command from partition 0, block 0." +#define LTFS31011D "state: CurrPosPart=%lld CurrPosBlck=%lld CurrPosFM=%lld Reserved=%d Locked=%d Ready=%d." +#define LTFS31025E "Cannot space %s: tried to space over BOT." +#define LTFS31005E "Cannot read: unit not ready." +#define LTFS31022D "Erase partition %lu." +#define LTFS31017E "Device already reserved." +#define LTFS31015E "Cannot format: unknown format type." +#define LTFS31003I "Closing device through generic itdtimage driver (%s)." +#define LTFS31196D "Backend %s: %llu." +#define LTFS12035E "Cannot rewind medium: backend call failed (%d)." +#define LTFS10006W "Null argument (%s) to %s." +#define LTFS10012E "Failed to register messages with libltfs (%d)." +#define LTFS11210W "Ignoring unexpected file mark in partition %lu." +#define LTFS17123E "Unexpected genaration value (Gen = %d, MAM IP = %d, MAM DP = %d)." +#define LTFS11245E "Cannot convert UTF-16 to UTF-8: failed to fill output buffer (%d)." +#define LTFS17133E "Failed to unload the cartridge." +#define LTFS11299E "Cannot format: unsupported medium." +#define LTFS11338I "Syncing index of %s %s." +#define LTFS11122E "Cannot set extended attribute: failed to look up the xattr (%d)." +#define LTFS11258I "No index found in the data partition." +#define LTFS17067D "Sync is kicked. (%s)." +#define LTFS11285E "Cannot load %s plug-in \'%s\': failed to load the message bundle (%d)." +#define LTFS11189E "Comparing labels: partition map mismatch." +#define LTFS11025D "Volume is consistent." +#define LTFS11217E "Cannot check medium: failed to write a file mark to the data partition (%d)." +#define LTFS11207E "Missing required index partition back pointer." +#define LTFS12012E "Cannot open device: failed backend open call." +#define LTFS12043E "Cannot write block: device is read-only." +#define LTFS17004E "XML parser: tag \'%s\' must not be empty." +#define LTFS12061W "Cannot get VCI data: unexpected Application Specific Info length %d." +#define LTFS11052E "Cannot unlink: path lookup failed (%d)." +#define LTFS11260E "Plug-in \'%s\' was not found in the configuration file." +#define LTFS11235E "Cannot validate name: failed to iterate code point." +#define LTFS11048E "Cannot create: failed to format the path (%d)." +#define LTFS11095E "Cannot format: the medium is write protected." +#define LTFS10020E "Error on %s: %s (%d, %d)." +#define LTFS17097E "XML parser: two extents overlap." +#define LTFS17087I "Kernel version: %s." +#define LTFS11219E "Cannot check medium: pointer verification failed (%d)." +#define LTFS17241W "Failed to open the advisory lock '%s' (%d)." +#define LTFS17255I "Cannot open %s cache for sync (%d)." +#define LTFS11153E "Cannot parse index criteria: failed to parse name rule (%d)." +#define LTFS12056W "Cannot get Volume Change Reference parameter: read attribute failed (%d)." +#define LTFS11195W "This index belongs to a different volume." +#define LTFS17052E "Cannot generate index data (%d) in file \'%s\'." +#define LTFS11000E "Cannot instantiate LTFS volume: failed to allocate device data." +#define LTFS11055E "Cannot rename: failed to format the destination path (%d)." +#define LTFS9012E "Cannot specify \"--quiet\" with \"--trace\" or \"--fulltrace\"." +#define LTFS17057E "Index writer: failed to start the document (%d)." +#define LTFS11251E "Cannot convert UTF-8 to system locale: failed to fill output buffer (%d)." +#define LTFS11345E "Failed to freeze tape %s because of permanent write errors on the IP and DP." +#define LTFS17104E "Cannot get PEWS: Mode Sense for Device Configuration Extension failed (%d)." +#define LTFS17169I "Detected EOD on the missing EOD cartridge." +#define LTFS11187E "Comparing labels: partition IDs must be lowercase ASCII characters." +#define LTFS17018E "XML parser: unsupported encoding \'%s\'." +#define LTFS11165E "Failed to pop a file name from a null stack pointer." +#define LTFS11087E "Cannot read: expected %u bytes from cache, but received %lu bytes." +#define LTFS11197E "Cannot read index: back pointer is corrupt." +#define LTFS11070E "Cannot write blocks: locate append position failed on partition %c." +#define LTFS11317E "Cannot print help message: unknown type: '%s'." +#define LTFS17074I "Device Name = %s, Vendor ID = %s, Product ID = %s, Serial Number = %s, Product Name =%s." +#define LTFS11068E "Cannot write blocks: multiple repetitions specified with an irregular buffer size." +#define LTFS11307W "All unique IDs for volume %s are used. No new files or directories can be created for the volume." +#define LTFS17201I "Starting a long wipe operation. This can take over 3 hours." +#define LTFS17005E "XML parser: extra content in tag \'%s\'." +#define LTFS17156E "Cannot get WRITE MODE: Mode Sense for Device Configuration Extension failed (%d)." +#define LTFS11127E "Cannot get extended attribute: failed to look up the path (%d)." +#define LTFS17160I "Maximum device block size is %d." +#define LTFS11071E "Cannot write blocks: failed to determine medium position (%d)." +#define LTFS11077E "Cannot write: failed to write blocks to the medium (%d)." +#define LTFS11137E "Cannot remove extended attribute: failed to format the name (%d)." +#define LTFS10001E "Memory allocation failed (%s)." +#define LTFS11102E "Cannot format: failed to write ANSI label (%d) on partition %u." +#define LTFS12053E "Cannot format medium: backend call failed (%d)." +#define LTFS17134E "Failed to reload the cartridge." +#define LTFS17142E "Both EODs are missing." +#define LTFS11277W "Cannot find the %s plug-in \'%s\' at \'%s\'." +#define LTFS11180E "Cannot read partition label: failed to space forward over the trailing file mark (%d)." +#define LTFS17219W "%s is out of range in parsing the index (%s)." +#define LTFS17011E "Cannot instantiate an index parser for file \'%s\'." +#define LTFS17119E "Failed to seek to the final index in %s (%d)." +#define LTFS17251I "File %s (%lld) was opened when this index was written." +#define LTFS17197W "Cannot retrieve the tape attribute data: unexpected length 0x%04x." +#define LTFS11238E "Cannot apply NFC: failed to get output buffer size (%d)." +#define LTFS11115W "Cache manager: failed to fully expand the pool." +#define LTFS12054E "Cannot unformat medium: locate to partition 0, block 0 failed (%d)." +#define LTFS12022I "Unloading medium." +#define LTFS11999E "Cannot load the medium: failed to get capacity data (%d)." +#define LTFS11265E "Cannot parse configuration file: \'default\' directive must be followed by a plug-in type and name." +#define LTFS11255I "Appending a file mark to the data partition." +#define LTFS11045E "Cannot set times: device is not ready." +#define LTFS12064E "Cannot write block: no space left on device." +#define LTFS11222E "Cannot check medium: failed to save index partition append position (%d)." +#define LTFS12031E "Cannot set compression: backend call failed (%d)." +#define LTFS17187E "Unexpected not ready state (%d)." +#define LTFS11090W "Data partition identifier requested on an uninitialized volume." +#define LTFS17084E "XML parser: failed to read extent list from file (%d)." +#define LTFS17258E "A critical error happened while parsing an index (%d). Stop seeking the latest index." +#define LTFS12048E "Cannot read: must open the device first." +#define LTFS11019D "Checking volume consistency." +#define LTFS11150E "Size= rule must contain a valid size." +#define LTFS17242W "Failed to acquire the advisory lock '%s' (%d)." +#define LTFS17054E "Cannot instantiate an index writer direct to tape." +#define LTFS11005I "Mounting the volume." +#define LTFS17086W "Cannot get kernel version." +#define LTFS11295E "Cannot read partition label: check for file mark after the ANSI label failed (%d)." +#define LTFS11316E "Failed to print help message of key manager interface backend." +#define LTFS17041E "XML parser: read failed while looking for a file mark (%d)." +#define LTFS17059E "Index writer: cannot validate extended attribute value (%d)." +#define LTFS11281E "Cannot load messages: failed to get message table (%d)." +#define LTFS17069E "Failed to sync index." +#define LTFS11029E "Cannot mount volume: failed to save the append position for the index partition." +#define LTFS17183E "Error writing XML schema to file '%s' on the disk." +#define LTFS12036E "Seek failed: final position is not what was requested." +#define LTFS11340I "Revalidation process is successfully done. (%s)" +#define LTFS11242E "Cannot convert UTF-8 to UTF-16: failed to get output buffer size (%d)." +#define LTFS11213E "Cannot check medium: seek index failed on the data partition (%d)." +#define LTFS17176E "Cannot read: failed to get data key (%d)." +#define LTFS11148E "More than one non-numeric character follows the size criterion." +#define LTFS17036E "XML parser: expected a text node (received type %d)." +#define LTFS17024E "XML parser: invalid size criterion %s." +#define LTFS17208I "Encountered the last data on the tape (%d). The last read index generation is %d." +#define LTFS11231E "A simple fix is possible to restore the consistency of the tape." +#define LTFS17170E "Failed to parse LTFS dcache configuration rules: invalid option '%s'." +#define LTFS11099E "Cannot format: failed to set the medium compression (%d)." +#define LTFS11225E "Cannot check medium: failed to allocate index data (%d)." +#define LTFS17093E "XML parser: failed to skip tag." +#define LTFS11177E "The ANSI label is too long (%d)." +#define LTFS11030I "Failed to sync volume (%d). Stop the periodic sync thread." +#define LTFS17083E "Traverse(%c): cannot locate to the last index position (%c)." +#define LTFS11147E "Duplicate index criteria option \'%s\'." +#define LTFS17230W "The bar code information is null." +#define LTFS17064D "Sync is done. (%s)." +#define LTFS11325E "Cannot set extended attribute: failed to flush (%d)." +#define LTFS11157E "Cannot specify a name rule without a size rule." +#define LTFS11083E "Cannot write index: failed to generate and write XML data (%d). The medium might be in an inconsistent state; unmount and run ltfsck." +#define LTFS17226W "Cannot update the tape attribute: the maximum (%s) attribute length is %d." +#define LTFS11331E "Failed to load the cartridge (%s)." +#define LTFS17245E "XML writer: failed to flush cached data to the disk (%d)." +#define LTFS11051E "Cannot unlink: failed to format the path (%d)." +#define LTFS11111E "Base64 decoder: empty input." +#define LTFS12026D "Locking medium in the drive." +#define LTFS17199W "Cannot update the tape attribute (type: 0x%04x): %s." +#define LTFS17100E "XML parser: UID on the root directory must be 1." +#define LTFS11183E "Comparing labels: volume UUID mismatch." +#define LTFS17014E "Cannot parse index: failed to create XML parser input buffer." +#define LTFS11128E "Cannot get extended attribute: failed to look for virtual xattrs (%d)." +#define LTFS11193E "Cannot read index: failed to determined medium position (%d)." +#define LTFS17002E "XML parser: duplicate optional tag \'%s\'." +#define LTFS11303E "Data placement rule is too long: '%s'." +#define LTFS17020E "XML parser: invalid schema version \'%s\'." +#define LTFS12018E "Cannot load the medium: failed to lock the medium in the drive (%d)." +#define LTFS17148E "Use ltfsck with the --deep-recovery option." +#define LTFS17098I "Device Name = %s (%d.%d.%d.%d), Vendor ID = %s, Product ID = %s, Serial Number = %s, Product Name =%s." +#define LTFS12021E "Cannot load the medium: failed to get device parameters (%d)." +#define LTFS11160E "Cannot prepare glob cache: failed to prepare name for caseless matching (%d)." +#define LTFS17264I "The index on %s is newer, but MAM shows a permanent write error happened on %s." +#define LTFS17152E "Cannot set WRITE MODE: Failed to load medium (%s, %d)." +#define LTFS11203E "Cannot locate index: failed to space back 2 file marks (%d)." +#define LTFS17126E "Unexpected EOD status (%d, %d)." +#define LTFS12016E "No medium present." +#define LTFS10005E "Null argument (%s) to %s." +#define LTFS17130I "Expected read error occurred." +#define LTFS17146E "EOD of %s(%d) is missing. A deep recovery operation is required." +#define LTFS17190I "Data key is set to enable encryption feature." +#define LTFS17109E "Failed to detect the final index or the final record." +#define LTFS11022I "Restoring volume consistency by writing an index to the index partition." +#define LTFS17263I "Sleep failed while taking the advisory lock '%s' (%d, %d)." +#define LTFS11221E "Medium check failed: detected invalid extents." +#define LTFS12046E "Cannot write file marks: must open the device first." +#define LTFS11204D "No index found. Space back and try again." +#define LTFS17136E "Failed to erase at EOD recovery point." +#define LTFS11272E "Cannot parse configuration file: \'option\' directive must be followed by an option type and LTFS mount option." +#define LTFS11105E "Cannot format: failed to generate LTFS label." +#define LTFS12050E "Cannot format medium: locate to partition 0, block 0 failed (%d)." +#define LTFS11321E "Symbolic link might be replaced by data file. Use ltfsck for recovery." +#define LTFS17096W "The index read from the tape uses a newer version of the LTFS format than the one supported by this software. Some functionality might be unavailable. If this tape is modified, the index downgrades format version to %s from %d.%d.%d." +#define LTFS11012D "Loading the tape." +#define LTFS12060W "Cannot get VCI data: unexpected VCR length 0x%04x." +#define LTFS11008D "Reading partition labels." +#define LTFS17021E "XML parser: unsupported %s version %s." +#define LTFS11261E "Cannot load plug-in: %s." +#define LTFS11049E "Cannot create: path lookup failed (%d)." +#define LTFS11226I "Erasing bad blocks from the index partition." +#define LTFS17206E "Cannot write XML data to file descriptor (%s, %d, %lu)." +#define LTFS11218E "Cannot check medium: failed to write a file mark to the index partition (%d)." +#define LTFS11142E "Cannot list physical xattrs: failed to convert key to system locale (%d)." +#define LTFS17254E "This cartridge cannot be reformatted in the drive (0x%02x, %d)." +#define LTFS11154E "Cannot parse index criteria: error while seeking name rule." +#define LTFS11054E "Cannot rename: failed to format the source path (%d)." +#define LTFS17053E "Cannot generate index data direct to tape: failed to create output buffer." +#define LTFS11291I "Eject successful." +#define LTFS11001E "Cannot instantiate LTFS volume: failed to allocate label data." +#define LTFS17246E "Failed to %s (%d)." +#define LTFS9013E "Cannot specify \"--quiet\" with \"--trace\" or \"--fulltrace\"." +#define LTFS17045E "Label writer: failed to format time." +#define LTFS11039E "Cannot open file: failed to format the path (%d)." +#define LTFS11046E "Cannot set read-only flag: device is not ready." +#define LTFS11088E "Cannot read: failed to get a block from the medium (%d)." +#define LTFS17164E "Cannot reset the capacity proportion: backend call failed (%d)." +#define LTFS17065D "Periodic sync thread initialized." +#define LTFS17113I "Starting EOD recovery: reading an index from (%llu, %llu)." +#define LTFS11194W "Cannot read index: failed to read and parse XML data (%d)." +#define LTFS17032E "XML parser: compression must be \'true\' (1) or \'false\' (0)." +#define LTFS11229E "Cannot restore medium consistency: failed to save index partition append position (%d)." +#define LTFS11310W "Cannot set the traversal mode: failed to get a valid selection %d." +#define LTFS17202E "Failed to wipe the medium (%d)." +#define LTFS17173W "Time conversion is underflowed. (%04d-%02d-%02dT%02d:%02d:%02dZ)." +#define LTFS11173E "Cannot read label (%d): failed to seek to block 0 (partition %lu)." +#define LTFS17207E "Use the ltfsck command with the --salvage-rollback-points option and select the latest index from the list\n Then use the ltfs command with the -o rollback-mount-no-eod option by specifying the generation." +#define LTFS11130E "Cannot list extended attributes: NULL output buffer with a non-zero size." +#define LTFS11319E "Cannot add the key to hash table at %s (%d)." +#define LTFS17234W "The index read from the tape uses a newer version of the LTFS format than the one supported by this software. If this tape is modified, the index downgrades to format version %s from %d.%d.%d." +#define LTFS10000E "Failed to initialize libltfs (%d)." +#define LTFS12013E "Cannot inquire the tape device: backend call failed (%d)." +#define LTFS17145W "Tape backend does not support missing EOD detection." +#define LTFS17217W "%s is out of range in setting time (%s:%llu sec=%lld)." +#define LTFS17135E "Failed to seek to EOD recovery point." +#define LTFS11101E "Cannot format: failed to locate (%d) to partition %u, block 0." +#define LTFS11098E "Cannot format: failed to partition the medium (%d)." +#define LTFS17159I "Process was interruped by user." +#define LTFS17129W "Unexpected read error occurred." +#define LTFS11276W "Ignoring unknown configuration directive \'%s\'." +#define LTFS17010E "Cannot parse XML label from memory." +#define LTFS11248E "Cannot convert system locale to UTF-16: failed to get output buffer size (%d) for '%s'." +#define LTFS11335D "Get physical block position (%d - %d)." +#define LTFS17092E "Index writer: failed to write opaque tags (%s)." +#define LTFS17071I "Unpartitioning the medium." +#define LTFS17003E "XML parser: cannot determine whether tag is empty." +#define LTFS12055E "Cannot unformat medium: backend call failed (%d)." +#define LTFS11271E "Cannot parse configuration file: unknown plug-in type \'%s\' in \'-default\' directive." +#define LTFS11108E "Cannot format: failed to write file mark after XML label (%d) on partition %u." +#define LTFS11116E "Cache manager: failed to grow the pool." +#define LTFS12025D "Releasing device." +#define LTFS11250E "Cannot convert UTF-8 to system locale: failed to get output buffer size (%d)." +#define LTFS11269E "Cannot parse configuration file: line too long." +#define LTFS12032E "Cannot set append position: invalid partition %lu." +#define LTFS11246E "Cannot convert system locale to UTF-16: failed to open converter (%d)." +#define LTFS17051E "Cannot instantiate an index writer to file \'%s\'." +#define LTFS12004E "Device backend does not implement all required methods." +#define LTFS17193I "Stack[%02d]: %p, %s." +#define LTFS10009E "No driver plug-in specified and no default found." +#define LTFS11006E "Cannot read volume: failed to load the tape." +#define LTFS11298E "Cannot read volume: unsupported medium." +#define LTFS11286E "Cannot load messages: failed to open resource bundle (%d)." +#define LTFS11188E "Comparing labels: partition IDs must be distinct." +#define LTFS11216E "Cannot check medium: failed to locate to EOD on the index partition (%d)." +#define LTFS17186E "Called releaseread_mrsw with zero or less count." +#define LTFS17194I "Stack[%02d]: %p." +#define LTFS17044E "Label writer: failed to start the document (%d)." +#define LTFS12042E "Cannot write block: must open the device first." +#define LTFS11026I "Performing a full medium consistency check." +#define LTFS11200E "Cannot locate index: failed to determined medium position (%d)." +#define LTFS17177E "Cannot read: no correct key." +#define LTFS11069E "Cannot write blocks: failed to write an index while switching partitions (%d)." +#define LTFS17031E "XML parser: invalid %s \'%s\'." +#define LTFS9008E "Cannot open output converter (ucnv_open: %d)." +#define LTFS17116I "Detecting the final record in DP." +#define LTFS10013E "Failed to set up signal handler." +#define LTFS11339D "%s volume lock status (%d)." +#define LTFS11059E "Cannot truncate: length must not be negative." +#define LTFS11230I "Writing index(es) to restore consistency." +#define LTFS11016W "The index partition MAM parameter is not usable. Mounting will take longer." +#define LTFS17227I "Tape attribute: %s = %s." +#define LTFS17224W "%s is out of range in creating the index (%lld)." +#define LTFS11174E "Cannot read ANSI label: read failed (%d)." +#define LTFS17237E "WORM related error (%s)." +#define LTFS11146E "Invalid index criteria option \'%s\'." +#define LTFS17079E "Traverse: cannot find target generation %d." +#define LTFS17250I "Information of %lld files are written to the sync list." +#define LTFS11324D "Path changed from '%s' to '%s'." +#define LTFS11050E "Cannot unlink: device is not ready." +#define LTFS11186E "Comparing labels: compression mismatch." +#define LTFS17167E "Cannot read volume: failed to get capacity data (%d)." +#define LTFS11164E "Failed to push file name onto a null stack pointer." +#define LTFS17107E "Version mismatch of MAM, IP=%d, DP=%d." +#define LTFS17017E "XML parser: unexpected top-level tag \'%s\'." +#define LTFS17117E "Failed to search the final index in DP." +#define LTFS11084E "Cannot write index: failed to write a trailing file mark (%d). The medium is in an inconsistent state; unmount and run ltfsck." +#define LTFS11190E "Comparing labels: unknown partition ID." +#define LTFS11306E "Cannot get read-only state of partition: invalid partition identifier." +#define LTFS11179E "Cannot read LTFS label: parsing failed (%d)." +#define LTFS11314E "Cannot format: failed to get data key (%d)." +#define LTFS11158E "Cannot match name: failed to initialize glob cache (%d)." +#define LTFS11126E "Cannot get extended attribute: failed to validate the name (%d)." +#define LTFS17157I "Changing the drive setting %s." +#define LTFS11309E "Cannot parse configuration file: \'-plugin\' directive must be followed by a plug-in type and name." +#define LTFS17229D "Truncate the tape attribute: %s (%s) to %d bytes." +#define LTFS11243E "Cannot convert UTF-8 to UTF-16: failed to fill output buffer (%d)." +#define LTFS12058W "Cannot get VCI data: unexpected ID 0x%04x." +#define LTFS17248W "Failed to open %s (%s)." +#define LTFS17222W "%s is out of range in getting xattr (%s:%llu sec = %lld)." +#define LTFS10004E "Cannot open device \'%s\'." +#define LTFS17125E "Failed to seek to the final record in %s (%d)." +#define LTFS12017E "Cannot load the medium (%d)." +#define LTFS17072E "Cannot unformat: failed to unpartition the medium (%d)." +#define LTFS17182D "Writing schema to disk: UUID='%s', barcode='%s'." +#define LTFS17131I "Recovering EOD at (%llu, %llu) takes about 20 minutes." +#define LTFS17139I "Starting a deep recovery operation: missing EOD." +#define LTFS17027E "XML parser: unsupported extended attribute type \'%s\'." +#define LTFS11033E "Cannot unmount: failed to write an index." +#define LTFS17108E "Cannot find partition id \'%c\'(0x%x)." +#define LTFS17047E "Cannot generate LTFS label: failed to allocate buffer." +#define LTFS17078D "Traverse(%c): cannot find target generation %d (%c)." +#define LTFS11023E "Cannot mount volume: seek to index failed on the index partition." +#define LTFS17185E "Cannot read LTFS label: max transfer length is shorter than max LTFS label length (%d)." +#define LTFS11282E "Cannot load messages: failed to determine first message ID (ures_getByKey: %d)." +#define LTFS11205E "Index partition contains a back pointer, but no index found on the data partition." +#define LTFS12045E "Cannot write block: backend call failed (%d). Dropping to read-only mode." +#define LTFS12010E "Failed to grab the device lock (%s)." +#define LTFS12051E "Cannot format medium: Mode Sense for Medium Partition failed (%d)." +#define LTFS17191I "Data key is cleared to disable encryption feature." +#define LTFS11275E "Cannot parse configuration file: \'plug-in\' directive must be followed by a plug-in type, name, and library path." +#define LTFS17055E "Cannot generate index data direct to tape (%d)." +#define LTFS17022E "XML parser: invalid block size %s." +#define LTFS11223I "Generating lost and found directory." +#define LTFS11093E "Cannot format: failed to load the medium (%d)." +#define LTFS11234E "Cannot validate extended attribute value: code point iteration failed." +#define LTFS11262E "Cannot unload plug-in: %s." +#define LTFS17049E "Cannot instantiate index writer to memory." +#define LTFS17220W "%s is out of range in parsing dentry (%s:%llu %s)." +#define LTFS11332I "Load successful." +#define LTFS11254D "libltfs read from \'%s\': offset = %lld, count = %llu." +#define LTFS11010E "Cannot read volume: failed to set medium compression." +#define LTFS17228I "Tape attribute: %s = 0x%02x." +#define LTFS17085I "Plugin: Loading \"%s\" %s backend." +#define LTFS11151E "Size= rule must contain a digit." +#define LTFS11018D "Done reading MAM parameters." +#define LTFS17007E "Cannot instantiate an LTFS label parser for file \'%s\'." +#define LTFS11002E "Cannot instantiate LTFS volume: failed to allocate index data." +#define LTFS9010E "Invalid option \'%s\'." +#define LTFS11228E "Cannot restore medium consistency: failed to save data partition append position (%d)." +#define LTFS12037E "Cannot seek: backend call failed (%d)." +#define LTFS11343I "Try to write an index on the IP on %s because of a permanent write error on the DP.." +#define LTFS11017W "The data partition MAM parameter is not usable. Mounting will take longer." +#define LTFS11253E "No index found in the medium." +#define LTFS11212E "Cannot create lost and found directory: seek failed (%d)." +#define LTFS17077I "Traverse(%c): find target generation %d (%c)." +#define LTFS11072E "Cannot write blocks: failed to write to the medium (%d)." +#define LTFS11089E "Cannot read: expected %u bytes from the medium, but received %u bytes." +#define LTFS17060E "XML writer: failed to write a block to the medium (%d)." +#define LTFS17110I "The MAM was written by PGA1 or earlier." +#define LTFS11305E "Empty data placement rule in '%s'." +#define LTFS11138E "Cannot remove extended attribute: failed to validate the name (%d)." +#define LTFS11311E "Cannot format: failed to reset capacity proportion (%d)." +#define LTFS17035E "XML parser: expected a text node." +#define LTFS17068I "Syncing index of %s (Reason: %s) %s." +#define LTFS11329E "Failed to recover tape: cannot write the index to an invalid position in the data partition (%lld, %lld, %d)." +#define LTFS17029E "XML parser: invalid UUID %s." +#define LTFS17088W "Cannot get distribution information." +#define LTFS11170E "Failed to read label (%d) from partition 0." +#define LTFS11125E "Cannot get extended attribute: failed to format the name (%d)." +#define LTFS17203W "Truncate the tape attribute: %s (%s) to %d bytes." +#define LTFS11131E "Cannot list extended attributes: failed to format the path (%d)." +#define LTFS17162I "Trying a simple recovery that requires several minutes to complete." +#define LTFS17174E "Key manager interface backend does not implement all required methods." +#define LTFS11104E "Cannot format: failed to write file mark after ANSI label (%d) on partition %u." +#define LTFS17140E "Tape backend does not support missing EOD detection." +#define LTFS11112E "Base64 decoder: invalid character in the input." +#define LTFS12029E "Device is not ready (%d)." +#define LTFS11182E "Comparing labels: tape bar code number mismatch." +#define LTFS17103E "Cannot set PEWS: Mode Select for Device Configuration Extension failed (%d)." +#define LTFS17189I "Device is becoming ready (%d)." +#define LTFS17013E "Cannot parse index: failed to determine medium position (%d)." +#define LTFS11080E "Cannot write index: failed to locate append position on partition %c (%d)." +#define LTFS17265I "Skip writing the index because of %s." +#define LTFS11302E "Data placement rule contains invalid character(s): '%s'." +#define LTFS12019E "Cannot load the medium: failed to determine medium position (%d)." +#define LTFS17118I "Seeking to the final index in %s (%llu, %llu)." +#define LTFS12020E "Cannot load the medium: failed to set device defaults (%d)." +#define LTFS17168E "Cannot read volume: medium is not partitioned." +#define LTFS11163E "Cannot perform matching: failed to open text boundary iterator for filename (%d)." +#define LTFS17046E "Label writer: failed to close the document (%d)." +#define LTFS12033E "Cannot seek to append position: seek failed (%d)." +#define LTFS17239E "Failed to update density (%s) %d." +#define LTFS11257I "No index found in the index partition." +#define LTFS11247E "Cannot convert system locale to UTF-16: failed to set up converter (%d)." +#define LTFS10008E "Failed to load the configuration file (%d)." +#define LTFS17121E "Failed to read the final index in %s (%d)." +#define LTFS11007D "Tape is loaded." +#define LTFS17233E "Failed to kick gcore." +#define LTFS17061E "XML writer: failed to flush cached data to the medium (%d)." +#define LTFS11297E "Cannot read index: failed to allocate index structure (%d)." +#define LTFS12038E "Cannot seek EOD: invalid partition %lu." +#define LTFS11287E "Cannot load messages: failed to register message data (%d)." +#define LTFS17238I "WORM status updated (%s=>%d) '%s'." +#define LTFS11215E "Cannot check medium: failed to locate to EOD on the data partition (%d)." +#define LTFS11027E "Cannot mount volume: medium consistency check failed." +#define LTFS17181E "Cannot reopen device: failed backend reopen call." +#define LTFS11240E "Cannot apply NFD: failed to get output buffer size (%d)." +#define LTFS11201E "Cannot locate index: failed to space back 1 file mark (%d)." +#define LTFS17195E "Cannot get maximum block size : %s (%d)." +#define LTFS17034E "XML parser: invalid time \'%s\' (%d)." +#define LTFS11320E "Cannot search the key from hash table at %s (%d)." +#define LTFS17095W "The index read from the tape uses an old version of the LTFS format. If this tape is modified, the index upgrades format version to %s from %d.%d.%d." +#define LTFS12063W "Cannot set VCI data: backend call failed (%d)." +#define LTFS10014E "Failed to clean up signal handler." +#define LTFS11009E "Cannot read volume: failed to read partition labels." +#define LTFS11237E "Cannot fold string case: failed to fill output buffer (%d)." +#define LTFS11015W "VCR MAM parameter is not usable. Mounting will take longer." +#define LTFS11227I "Preserve existing unreferred data blocks in the data partition, put the latest index at %llu." +#define LTFS17080D "Traverse(%c): find generation %d (%c)." +#define LTFS17026E "XML parser: file size is shorter than extent list." +#define LTFS17081E "Traverse(%c): callback function failed %d (%c)." +#define LTFS11097I "Partitioning the medium." +#define LTFS17257I "0x2f (/) or 0x1f (US) exists in name object (%s): Replace it with '_'." +#define LTFS11141E "Cannot list physical xattrs: failed to generate namespace prefix (%d)." +#define LTFS17247W "Failed to create the path of %s (%s)." +#define LTFS11155E "Cannot parse index criteria: failed to parse size rule (%d)." +#define LTFS17066D "Periodic sync thread uninitialized." +#define LTFS11327E "Failed to seek EOD: seek invalid partition (%d, %d)." +#define LTFS11057E "Cannot rename: path lookup failed for destination (%d)." +#define LTFS11337I "Update index-dirty flag (%d) - %s (0x%p)." +#define LTFS11267E "Cannot get library path: unknown plug-in type \'%s\' or plug-in name \'%s\'." +#define LTFS17050E "Cannot generate index data in memory." +#define LTFS17008E "Cannot parse XML label from file \'%s\'." +#define LTFS11047E "Cannot create: device is not ready." +#define LTFS11290E "Failed to eject the cartridge (%s)." +#define LTFS11117E "Cannot set extended attribute: device is not ready." +#define LTFS17073I "Tape Device list:." +#define LTFS11185E "Comparing labels: block size mismatch." +#define LTFS17016E "Cannot parse index direct from medium." +#define LTFS11167E "Cannot create file or directory: failed to allocate dentry." +#define LTFS17106E "XML parser: UID 0 is reserved." +#define LTFS11191E "Comparing labels: partitions have the same ID \'%c\'." +#define LTFS17232E "Failed to initialize file system component (%d)." +#define LTFS11085E "Cannot read: failed to determine medium position (%d)." +#define LTFS11301E "Unable to read index: cannot duplicate index partition criteria." +#define LTFS17179I "Cleaning cartridge was loaded. (TA=%016llx)." +#define LTFS13003E "Scheduler backend does not implement all required methods." +#define LTFS17114I "Seaching the final index in IP." +#define LTFS11315E "Cannot format: failed to set data key (%d)." +#define LTFS11344I "Tape %s is frozen successfully because of a permanent write error on the DP." +#define LTFS17039E "XML parser: failed to read a block from the medium (%d)." +#define LTFS11118E "Cannot set extended attribute: failed to format the path (%d)." +#define LTFS17154E "Cannot set WRITE MODE: Mode Sense for Device Configuration Extension failed (%d)." +#define LTFS11308E "Commit message must be 65536 bytes or less." +#define LTFS17166D "Coherency Data: %s (%lld, %lld, %lld, %u, %s, %c)." +#define LTFS11121E "Cannot set extended attribute: failed to look up the path (%d)." +#define LTFS12041E "Cannot space file marks: backend call failed (%d)." +#define LTFS11100I "Writing label to partition %c." +#define LTFS10003E "Cannot initialize condition variable (%d)." +#define LTFS12014E "Cannot open device: failed to reserve the device (%d)." +#define LTFS11279E "Cannot write index to partition %c (%d)." +#define LTFS11214E "Cannot check medium: seek index failed on the index partition (%d)." +#define LTFS17165I "Resetting the medium's capacity proportion." +#define LTFS17188I "Device is not ready (%d)." +#define LTFS17144E "The MAM of %s is not usable." +#define LTFS11336I "The attribute does not exist. Ignore the expected error." +#define LTFS11249E "Cannot convert system locale to UTF-16: failed to fill output buffer (%d) for '%s'." +#define LTFS11020E "Cannot mount volume: seek to index failed on the data partition." +#define LTFS11198E "Cannot read index: failed to space forward 1 file mark (%d)." +#define LTFS11270E "Cannot parse configuration file: \'-default\' directive must be followed by a plug-in type." +#define LTFS17192W "Cannot read: medium is encrypted." +#define LTFS12024E "Cannot reserve device: backend call failed (%d)." +#define LTFS17023E "XML parser: invalid generation number %s." +#define LTFS11053E "Cannot rename: device is not ready." +#define LTFS11263E "Cannot resolve plug-in operation interface: %s." +#define LTFS10019E "Error on runcommand (%s, %d, %d)." +#define LTFS11123E "Cannot get extended attribute: NULL value with non-zero size." +#define LTFS11333I "A cartridge with write-perm error is detected on %s. Seek the newest index (IP Gen = %llu, VCR = %llu) (DP Gen = %llu, VCR = %llu) (VCR = %llu)." +#define LTFS10021D "xattr: %s: %s (%d, %d)." +#define LTFS11220E "Medium check failed: extra blocks detected. Run ltfsck." +#define LTFS17240I "Density code is updated to x%02x." +#define LTFS11152E "Cannot parse index criteria: rules are invalid." +#define LTFS11003E "Cannot retrieve device capacity data (%d)." +#define LTFS11268E "Cannot open configuration file \'%s\' (%d)." +#define LTFS9011E "Logging initialization failed." +#define LTFS11040E "Cannot open file: path lookup failed (%d)." +#define LTFS11293E "Cannot load messages for libltfs (%d)." +#define LTFS11252D "libltfs write to \'%s\': offset = %lld, count = %llu." +#define LTFS11342E "Failed to get the volume lock status (%d)." +#define LTFS12034E "Cannot get maximum device blocksize: backend call failed (%d)." +#define LTFS11283E "Cannot load messages: failed to determine first message ID (ures_getInt: %d)." +#define LTFS11334I "Remove extent : %s (%llu, %llu)." +#define LTFS11135E "Cannot remove extended attribute: device is not ready." +#define LTFS11312I "Revalidation process of the medium is starting. (%s)" +#define LTFS17178E "Cannot read: failed to set data key (%d)." +#define LTFS17030E "XML parser: failed to normalize %s \'%s\'." +#define LTFS17204W "Cannot set: unkown tape attribute type(0x%04x): %s." +#define LTFS17260I "Sleep was interrupted by a signal while taking the advisory lock '%s'." +#define LTFS11233I "Updating MAM coherency data." +#define LTFS17091E "Cannot save tag: libxml2 could not return text for this node." +#define LTFS11169E "Cannot read labels: failed to allocate label data (%d)." +#define LTFS17141I "Both EODs are detected. A deep recovery operation is unnecessary." +#define LTFS17236I "Wrote index of %s (%c, %s)." +#define LTFS9014E "Cannot create work directory \'%s\': %s." +#define LTFS11175E "Cannot read ANSI label: expected 80 bytes, but received %d." +#define LTFS17172W "Time conversion is overflowed. (%04d-%02d-%02dT%02d:%02d:%02dZ)." +#define LTFS17171E "Failed to parse LTFS dcache configuration rules: invalid value '%d' for option '%s'." +#define LTFS11145E "Cannot get attribute %s: failed to generate the time string." +#define LTFS11132E "Cannot list extended attributes: failed to look up the path (%d)." +#define LTFS17223W "%s is out of range in creating the label (%lld)." +#define LTFS11159E "Cannot match name: failed to prepare for caseless matching (%d)." +#define LTFS17253E "Cannot get tape parameters: %s (%d)." +#define LTFS11323D "Symlink EA prefix valuse is (%s)." +#define LTFS9006E "Cannot load resource \"fallback_messages\" (ures_getByKey: %d)." +#define LTFS17143I "EOD of %s(%d) is missing." +#define LTFS12028D "Unlocking medium." +#define LTFS9016E "Cannot set the LANG environment variable." +#define LTFS11113E "Base64 decoder: input length is not a multiple of 4." +#define LTFS11181E "Cannot read partition label: failed to find the trailing file mark." +#define LTFS17012E "Cannot parse index from file \'%s\'." +#define LTFS17102E "Cannot set PEWS: Mode Sense for Device Configuration Extension failed (%d)." +#define LTFS17266I "Skip setting the append only mode because the drive doesn't seem to support it." +#define LTFS11081E "Cannot write index: failed to determine medium position (%d)." +#define LTFS11998W "Unable to delete dentry '%s': it still has outstanding references." +#define LTFS17028E "XML parser: base64 decoding failed." +#define LTFS12023D "Reserving device." +#define LTFS11114E "Cache manager: failed to initialize the pool." +#define LTFS11211E "Cannot populate lost and found directory: failed to allocate file data." +#define LTFS17048E "Cannot generate index data in memory: failed to allocate buffer." +#define LTFS12059W "Cannot get VCI data: unexpected length 0x%04x." +#define LTFS11244E "Cannot convert UTF-16 to UTF-8: failed to get output buffer size (%d)." +#define LTFS10007E "Not yet implemented (%s)." +#define LTFS17249I "Information of %lld files are written to the offset cache." +#define LTFS17221W "%s is out of range in setting xattr (%s:%llu %s)." +#define LTFS17132E "Failed to get current position." +#define LTFS17124I "Seeking to the final record in %s (%llu, %llu)." +#define LTFS11032D "Unmounting the volume." +#define LTFS17196W "Cannot retrieve the tape attribute data: unexpected ID 0x%04x." +#define LTFS17184E "Error changing index cache file's permission (%d)." +#define LTFS11259I "Cannot move directory: Directory move is prohibited because of a MacFUSE bug." +#define LTFS11284E "Cannot resolve plug-in message bundle interface: %s." +#define LTFS11024E "Cannot mount volume: read index failed on the index partition." +#define LTFS17056E "XML writer: cannot format time (gmtime failed)." +#define LTFS11206E "Index partition back pointer is invalid." +#define LTFS17138I "Recovered EOD status successfully (%d)." +#define LTFS12044E "Cannot write a %u-byte block: maximum block size is %lu." +#define LTFS11014D "Reading MAM parameters." +#define LTFS17000E "XML parser: not all required tags found in \'%s\'." +#define LTFS12062W "Cannot get VCI data: unexpected Application Specific Info signature." +#define LTFS11236E "Cannot fold string case: failed to get output buffer size (%d)." +#define LTFS11096E "Cannot format: requested block size is %lu bytes, but the device only supports %u." +#define LTFS11224E "Cannot restore medium consistency: failed to generate lost and found (%d)." +#define LTFS17094E "XML parser: comment field is longer than 64 KiB." +#define LTFS17231E "Cannot %s the tape attribute: %s." +#define LTFS17082E "Traverse(%c): cannot locate to the first index position (%c)." +#define LTFS11176E "The ANSI label indicates this is not an LTFS volume." +#define LTFS11140E "Cannot remove extended attribute: failed to look up the xattr (%d)." +#define LTFS9015W "Setting the locale to 'en_US.UTF-8'. If this setting is wrong, set the LANG environment variable before starting %s." +#define LTFS11326E "Cannot write index: failed to flush buffered data (%d)." +#define LTFS11156E "Cannot parse index criteria: error while seeking size rule." +#define LTFS11330I "Loading cartridge." +#define LTFS17076E "Traverse(%c): cannot locate to the next index position (%c)." +#define LTFS9001E "Failed to parse command line options." +#define LTFS17244E "XML writer: failed to write a block to the disk (%d)." +#define LTFS11056E "Cannot rename: path lookup failed for source (%d)." +#define LTFS11264E "Cannot get plug-in operations." +#define LTFS17256I "An unexpected character exists in the decoded name object (%s). Revert to the encoded character." +#define LTFS11239E "Cannot apply NFC: failed to fill output buffer (%d)." +#define LTFS17019E "XML parser: no schema version found." +#define LTFS11162E "Cannot perform matching: failed to open text boundary iterator for criteria (%d)." +#define LTFS17105E "Cannot load the medium: failed to get programmable early warning size (%d)." +#define LTFS17111I "The MAM was written by PGA2 or later." +#define LTFS11304E "Failed to validate data placement rule (%d)." +#define LTFS11178E "Cannot read LTFS label: read failed (%d)." +#define LTFS11086E "Cannot read: failed to locate (%d) to partition %c, record %llu." +#define LTFS17225W "%s is out of range in creating dentry (%lld)." +#define LTFS11139E "Cannot remove extended attribute: failed to look up the path (%d)." +#define LTFS11196W "Cannot read index: self-pointer mismatch." +#define LTFS17101E "XML parser: UID 1 is reserved for the root directory." +#define LTFS11328E "Failed to seek index: seek invalid partition (%c, %c)." +#define LTFS17200E "XML parser: cannot save tag, libxml2 workaround failed (%s)." +#define LTFS11124E "Cannot get extended attribute: failed to format the path (%d)." +#define LTFS11171E "Failed to read label (%d) from partition 1." +#define LTFS17089I "Distribution: %s." +#define LTFS17161I "EOD information (%s) is corrupted." +#define LTFS17175E "Cannot read: failed to get data key identifier (%d)." +#define LTFS11149E "Invalid size criterion multiplier \'%c\'." +#define LTFS17151E "Cannot set WRITE MODE: Failed to unload medium (%d)." +#define LTFS11136E "Cannot remove extended attribute: failed to format the path (%d)." +#define LTFS11202E "Cannot locate index: failed to space forward 1 file mark (%d)." +#define LTFS11278I "Writing index to partition %c." +#define LTFS12040E "Cannot parse backend options: backend call failed (%d)." +#define LTFS17127I "Detecting the final record to read by hitting read error takes about 20 minutes." +#define LTFS10002E "Cannot initialize mutex (%d)." +#define LTFS11082E "Cannot write index: failed to write file mark (%d)." +#define LTFS11289I "Ejecting cartridge." +#define LTFS17137E "Failed to recover EOD status (%d)." +#define LTFS17147I "Attempting to mount a cartridge without EOD status check." +#define LTFS17006W "XML parser: ignoring unrecognized tag \'%s\' inside \'%s\'." +#define LTFS17198D "Cannot get the tape attribute: 0x%04x (%s)." +#define LTFS17037E "XML parser: failed to read from XML stream." +#define LTFS11021E "Cannot mount volume: read index failed on the data partition." +#define LTFS11199E "Cannot locate index: failed to locate to EOD (%d)." +#define LTFS12057W "Cannot get VCI data: read attribute failed (%d)." +#define LTFS12008E "Cannot allocate device data: failed to initialize mutex (%d)." +#define LTFS12047E "Cannot write file marks: backend call failed (%d). Dropping to read-only mode." +#define LTFS11273E "Cannot parse configuration file: \'%s\' directive must be followed by a valid %s name." +#define LTFS11106E "Cannot format: failed to write XML label (%d) on partition %u." +#define LTFS17218W "%s is out of range in parsing the label (%s)." +#define LTFS11346E "Tape %s is frozen by the previous index on the tape. Some metadata will be lost during the unmount operation." +#define LTFS11256I "Appending a file mark to the index partition." +#define LTFS12030E "Cannot get capacity data: backend call failed (%d)." +#define LTFS11091W "Index partition identifier requested on an uninitialized volume." +#define LTFS17120I "Reading the final index in %s (%llu, %llu)." +#define LTFS17259I "Recover an index on %s from (%c, %llu)." +#define LTFS11209E "Cannot create lost and found directory: failed to allocate directory data." +#define LTFS17062E "XML writer: tried to write a directory as a file." +#define LTFS12049E "Cannot read: backend call failed (%d)." +#define LTFS11004E "Cannot take the device lock (%s)." +#define LTFS11296E "Cannot read partition label: failed to find a file mark after the ANSI label." +#define LTFS17009E "Cannot instantiate an LTFS label parser for a memory buffer." +#define LTFS17058E "Index writer: failed to close the document (%d)." +#define LTFS17040E "XML parser: failed to space back 1 file mark." +#define LTFS17180E "File %s has both of symbolic link and extents." +#define LTFS12039E "Cannot seek EOD: backend locate call failed (%d)." +#define LTFS11280E "Unknown default %s \'%s\'." +#define LTFS11241E "Cannot apply NFD: failed to fill output buffer (%d)." +#define LTFS11028D "Consistency check finished." +#define LTFS17070I "Synced index of %s (%d) %s." +#define LTFS11313E "Medium revalidation failed (%d). Unmount the tape before continuing. (%s)" +#define LTFS11062E "Cannot truncate: device is not ready." +#define LTFS11067E "Cannot write blocks: invalid partition identifier." +#define LTFS17112I "Starting EOD recovery (GA/PGA1)." +#define LTFS17033E "XML parser: invalid partition \'%s\'." +#define LTFS11232E "Rerun the consistency check with simple fixes enabled." +#define LTFS11168E "Cannot allocate index data: failed to allocate root dentry." +#define LTFS17205E "Cannot set the tape attribute (type: 0x%04x): %s ." +#define LTFS17235I "Writing index of %s to %c (Reason: %s, %lld files) %s." +#define LTFS11318W "Unknown log level (%d), forced the level to (%d)." +#define LTFS17252W "Invalid value (%s) in the %s tag in UID %lld." +#define LTFS17063W "Periodic sync thread failed to flush file data to the medium. Data might be lost (%s)." +#define LTFS11172E "Cannot verify labels: comparison failed (%d)." +#define LTFS11133E "Cannot list extended attributes: failed to list real xattrs (%d)." +#define LTFS17243W "Failed to release the advisory lock (%d)." +#define LTFS17090W "sysctl system call failed (%s)." +#define LTFS11013D "Partition labels are valid." +#define LTFS11322D "Makeing a symlink '%s' target '%s'." +#define LTFS17042E "XML writer: error creating tag (%s)." +#define LTFS12027E "Cannot lock medium in the drive: backend call failed (%d)." +#define LTFS11110E "Cannot get current time (%d)." +#define LTFS11011E "Cannot read volume: block size is %lu, but the device only supports a block size of %u." +#define LTFS11184E "Comparing labels: format time mismatch." +#define LTFS17015E "Cannot parse index: failed to create XML reader." +#define LTFS11166E "Cannot allocate index data: failed to initialize mutex (%d)." +#define LTFS11288W "No end ID found for this message bundle: assigning 1000 message IDs." +#define LTFS11192E "Comparing labels: invalid bar code number." +#define LTFS17001E "XML parser: duplicate required tag \'%s\'." +#define LTFS17115E "Failed to search the final index in IP." +#define LTFS11129E "Cannot get extended attribute: failed to look up the xattr (%d)." +#define LTFS17043E "Cannot instantiate an LTFS label writer to memory." +#define LTFS11034I "Volume unmounted successfully." +#define LTFS13004E "Dentry cache backend does not implement all required methods." +#define LTFS17261I "The kernel detected a false deadlock retry to acquire the advisory lock '%s' (%d)." +#define LTFS17038E "XML parser: unexpected end of XML stream." +#define LTFS11119E "Cannot set extended attribute: failed to format the name (%d)." +#define LTFS17075E "Traverse(%c): cannot read index from %d (%c)." +#define LTFS17149E "Cannot erase: backend call failed (%d)." +#define LTFS17099E "Failed to spawn the periodic sync thread (%d)." +#define LTFS17155E "Cannot set WRITE MODE: Mode Select for Device Configuration Extension failed (%d)." +#define LTFS11161E "Cannot match name: match function failed (%d)." +#define LTFS11120E "Cannot set extended attribute: failed to validate the name (%d)." +#define LTFS13019I "FCFS I/O scheduler initialized." +#define LTFS13020I "FCFS I/O scheduler uninitialized." +#define LTFS30267W "WWPID of reservation: x%02x%02x%02x%02x%02x%02x%02x%02x (%s)." +#define LTFS30277W "Detect ENOMEM in ioctl() call. Wait 3 seconds and retry (%d)." +#define LTFS30222W "Received low space warning (early warning) in %s." +#define LTFS30218D "Read block: underrun in illegal length. residual = %d, actual = %d." +#define LTFS30283W "Skip back for %s operation is failed (%d). (%u, %llu), (%u, %llu)." +#define LTFS30247I "No alternate device is found for drive %s." +#define LTFS30210I "Cannot open device: failed to open %s (%d)." +#define LTFS30234I "Cannot get log page 0x%02x (%d) in %s." +#define LTFS30207I "Vendor ID is %s." +#define LTFS30257D "Transferring dump data." +#define LTFS30223W "Received low space warning (programmable early warning) in %s." +#define LTFS30276I "Raw capacity info of partition %d (%llu - %llu) %s." +#define LTFS30280W "Unexpected current position for %s operation (%d). (%u, %llu), (%u, %llu) ." +#define LTFS30204D "%s (0x%02x) expected error %d." +#define LTFS30244I "CDB unexpected status: H = 0x%02x, D = 0x02%x" +#define LTFS30238W "Cannot get EOD status: failed to parse the log page." +#define LTFS30213I "Unsupported Drive \'%s\' / \'%s\'." +#define LTFS30233I "Cannot read attribute (%d)." +#define LTFS30293I "Changer %s was reserved from another node (%s)." +#define LTFS30392D "Backend %s %s." +#define LTFS30268I "Retry to reserve from key registration (%s)." +#define LTFS30260W "Cannot retrieve drive dump: wrote %d bytes out, expected %d." +#define LTFS30256D "Total number of transfers is %d." +#define LTFS30228I "Unsupported cartridge (0x%x, 0x%x)." +#define LTFS30265W "Failed to get medium type code: medium type check is skipped." +#define LTFS30216W "Length mismatch is detected. (Act = %d, resid = %d, resid_sense = %d)." +#define LTFS30291I "Changer %s was reserved from this node and can be reserved from the current path." +#define LTFS30271I "Successfully reopened drive %s with the same path." +#define LTFS30201D "CDB check condition: sense = %06x, %s." +#define LTFS30249I "Opening another path for drive %s on %s." +#define LTFS30285I "The reserved buffer size of %s is %d." +#define LTFS30236I "Unexpected parameter size for getting active CQ loss write (%d)." +#define LTFS30209I "Opening a device through sg-ibmtape driver (%s)." +#define LTFS30220I "Drive cleaning required." +#define LTFS30241E "Invalid scsi_lbprotect option: %s." +#define LTFS30279I "Retrying the %s operation at (%u, %llu) with skip back, because the command was already run (%u, %llu)." +#define LTFS30397D "Backend %s: (%llu, %llu) %s." +#define LTFS30255D "Total dump data length is %lld." +#define LTFS30282W "Unexpected position after skip back for a %s operation. (%u, %llu), (%u, %llu)." +#define LTFS30399I "sg tape backend for IBM tape options:\n -o devname= tape device (default=%s)\n" +#define LTFS30266W "The drive is already reserved: %s (%s)." +#define LTFS30229I "Cannot get remaining capacity: get log page 0x%02x failed (%d)." +#define LTFS30290I "Changer %s isn't reserved from any nodes." +#define LTFS30270I "A power-on-reset happened on drive %s." +#define LTFS30219D "Read block: file mark detected." +#define LTFS30221E "Logical block protection Error on read." +#define LTFS30211I "Cannot %s blocking mode (%d)." +#define LTFS30235I "Cannot parse the log page 0x%02x in %s." +#define LTFS30206I "Cannot open device: inquiry failed (%d)." +#define LTFS30254W "Cannot retrieve drive dump: failed to create dump file (%d)." +#define LTFS30286I "Failed to get the cartridge status. The cartridge is not loaded." +#define LTFS30278I "Retrying the %s operation at (%u, %llu)." +#define LTFS30246I "Connection down is detected, try to reconnect (%s)." +#define LTFS30214I "Firmware revision is %s." +#define LTFS30263I "%s returns %s (%d) %s." +#define LTFS30287I "Received an unknown sense code %06x." +#define LTFS30230I "Cannot parse remaining capacity (0x%x, %d)." +#define LTFS30203I "CDB unexpected status: S = 0x%02x, M = 0x02%x" +#define LTFS30253I "Saving drive dump to %s." +#define LTFS30226I "A long data wipe is in progress. (%d minutes passed)." +#define LTFS30273I "Cannot %s device flag (%d)." +#define LTFS30243E "Encryption feature is not supported on the drive: %d." +#define LTFS30248I "Drive serial is not matched. Actual: %s, Expected %s." +#define LTFS30217I "Read block: overrun in illegal length. residual = %d, actual = %d." +#define LTFS30284I "Cannot get the reserved buffer size of %s." +#define LTFS30264E "%s returns msg = NULL (%d) %s." +#define LTFS30200I "Failed to execute SG_IO ioctl, opcode = %02x (%d)." +#define LTFS30208I "Product ID is %s." +#define LTFS30252I "Logical block protection is disabled." +#define LTFS30227I "A long data wipe is in progress. %d %%." +#define LTFS30272I "Drive %s is successfully reserved." +#define LTFS30398D "Backend %s: (%llu, %llu) FM = %llu %s." +#define LTFS30240I "Cannot open directory (/dev)." +#define LTFS30237W "Cannot get EOD status: failed to get log page 0x%02x (%d)." +#define LTFS30396D "Backend %s: %llu %s." +#define LTFS30394D "Backend %s: %zu %s." +#define LTFS30259W "Cannot retrieve drive dump: failed to write to dump file (%d)." +#define LTFS30232I "Cannot get remaining capacity: failed to parse the log page." +#define LTFS30205I "%s (0x%02x) returns %d." +#define LTFS30281W "Position confirmation for %s operation retry is failed (%d). (%u, %llu), (%u, %llu)." +#define LTFS30275I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." +#define LTFS30245D "Tape device returns %d, ignore for buffered sense cleaning." +#define LTFS30212I "Cannot get drive identifier of %s." +#define LTFS30261I "Taking drive dump in buffer." +#define LTFS30224D "EOD detected (%s): ignore sense." +#define LTFS30393D "Backend %s: %d %s." +#define LTFS30292I "Changer %s was reserved from this node but failed to reserve from the current path." +#define LTFS30269I "Successfully reopened drive %s with another path. Preempting the reservation." +#define LTFS30288I "Opening a tape device for drive serial %s." +#define LTFS30251I "Logical block protection is enabled." +#define LTFS30395D "Backend %s: %zu bytes %s." +#define LTFS30258W "Cannot retrieve drive dump: failed to read buffer (%d)." +#define LTFS30262I "Forcing drive dump." +#define LTFS30239W "Cannot get EOD status: value length or partition number is wrong %d - (%d, %d)." +#define LTFS30231I "Unexpected field in remaining capacity (0x%x)." +#define LTFS30202D "CDB %s." +#define LTFS30274I "Pseudo-error on %s." +#define LTFS30250I "Opened the SCSI tape device %d.%d.%d.%d (%s)." +#define LTFS30225I "Unrecognized space type." +#define LTFS30215I "Drive serial is %s." +#define LTFS30242E "Encryption method of the drive is not AME but %s (0x%02X)." +#define LTFS30289I "Cannot fetch the reservation key (%s, %d)." +#define LTFS14094E "Cannot get mount point (%d)." +#define LTFS14041D "FUSE mkdir \'%s\'." +#define LTFS14114E "Cannot initialize the open file table." +#define LTFS14046D "FUSE rename \'%s\' to \'%s\'." +#define LTFS14031D "FUSE getattr for \'%s\'." +#define LTFS14009E "The backend \'%s\' does not have a default device. Specify one using the \"-o devname=\" option." +#define LTFS14066E "Sync time should be a positive value." +#define LTFS14049D "FUSE read \'%s\' (offset=%lld, count=%zu)." +#define LTFS14039D "FUSE chmod \'%s\'." +#define LTFS14001E "Cannot enable FUSE option %s (%d)." +#define LTFS14422I " -o rules= Rules for choosing files to write to the index partition.\n" +#define LTFS14079E "Invalid uid \'%s\' (must be a positive integer or valid user name)." +#define LTFS14015W "Volume does not allow index criteria override. Ignoring user-specified criteria." +#define LTFS14410I " -o umask=M Set file permissions (octal)" +#define LTFS14026D "Cannot read directory: no buffer space." +#define LTFS14463I " -o scsi_append_only_mode= Set the tape device append-only mode (default=on)" +#define LTFS14115E "Invalid scsi_append_only_mode option: %s." +#define LTFS14057E "Failed to load kmi plug-in (%d)." +#define LTFS14443I " -o force_mount_no_eod Skip EOD existence check when mounting (read-only mount)\n" +#define LTFS14030D "FUSE fgetattr for \'%s\'." +#define LTFS14054E "Failed to load tape drive plug-in (%d)." +#define LTFS14077I "The cartridge will be mounted as read-only." +#define LTFS14067E "Failed to convert the sync time because it is too big or too small (%s)." +#define LTFS14413I " -o config_file= Configuration file (default: %s)" +#define LTFS14436I " -o device_list Show available tape devices" +#define LTFS14006E "Invalid umask \'%s\' (must be 3 octal digits, such as 022)." +#define LTFS14109E "Minimum pool size must be a positive number." +#define LTFS14038D "FUSE set times \'%s\'." +#define LTFS14423I " -o quiet Disable informational messages (same as verbose=1)" +#define LTFS14403I " -o devname= Tape device" +#define LTFS14078I "Medium is Read-Only in this device." +#define LTFS14112I "Invoke 'mount' command to check the result of final setup." +#define LTFS14072I "Rollback mount is specified. Mounting as read-only at generation %d." +#define LTFS14419I " -o dmask= Override directory permission mask (3 octal digits, default: 000)" +#define LTFS14043D "FUSE ftruncate \'%s\' (length=%lld)." +#define LTFS14461I " -o symlink_type= Specify symbolic link type (default: posix)\n" +#define LTFS14405I " -o trace Enable diagnostic output (same as verbose=3)" +#define LTFS14104I "Launched by \"%s\"." +#define LTFS14064I "Sync type is \"%s\"." +#define LTFS14052D "FUSE listxattr \'%s\'." +#define LTFS14033D "FUSE open directory \'%s\'." +#define LTFS14013E "Cannot mount the volume." +#define LTFS14074I "Data partition has no space to write index. Mounting as read-only." +#define LTFS14420I " -o min_pool_size= Minimum write cache pool size. Cache objects are 1 MB each (default: %d)" +#define LTFS14003E "min_pool_size (%d) cannot be greater than max_pool_size (%d)." +#define LTFS14416I " -o iosched_backend= I/O scheduler implementation to use (default: %s, use \"none\" to disable)" +#define LTFS14093E "Unknown type of symbolic link (%s)." +#define LTFS14024E "Cannot get extended attribute: position option must be zero." +#define LTFS14040D "FUSE create file \'%s\'." +#define LTFS14113I "Specified mount point is listed if succeeded." +#define LTFS14071I "Medium has no space to write data. Mounting as read-only." +#define LTFS14418I " -o fmask= Override file permission mask (3 octal digits, default: 000)" +#define LTFS14095I "Set the tape device write-anywhere mode to avoid cartridge ejection." +#define LTFS14008E "Invalid dmask \'%s\' (must be 3 octal digits, such as 022)." +#define LTFS14402I " -o devname= Tape device (default: %s)" +#define LTFS14061E "Unknown type of sync (%s)." +#define LTFS14032D "FUSE open file \'%s\' (%s)." +#define LTFS14055E "Failed to load I/O scheduler plug-in (%d)." +#define LTFS14089E "Could not initialize the kmi plug-in." +#define LTFS14065E "Unexpected sync type (%d)." +#define LTFS14012E "Tape backend option parsing failed." +#define LTFS14000I "LTFS starting, %s version %s, log level %d." +#define LTFS14421I " -o max_pool_size= Maximum write cache pool size. Cache objects are 1 MB each (default: %d)" +#define LTFS14456I " -o capture_index Capture latest index to work directory at unmount" +#define LTFS14411I " -o uid=N Set file owner" +#define LTFS14080E "Invalid gid \'%s\' (must be a positive integer or valid group name)." +#define LTFS14092I "Symbolic link type is (%s)." +#define LTFS14035D "FUSE release file \'%s\'." +#define LTFS14062I "Sync time is 0. Set sync type \"none\"." +#define LTFS14407I " -V, --version Output version information and exit" +#define LTFS14019I "Medium is write protected. Mounting read-only." +#define LTFS14106I "GCC version is %s." +#define LTFS14058I "%s version %s." +#define LTFS14045D "FUSE rmdir \'%s\'." +#define LTFS14110E "Maximum pool size must be a positive number." +#define LTFS14048D "FUSE write \'%s\' (offset=%lld, count=%zu)." +#define LTFS14050D "FUSE setxattr \'%s\' (name=\'%s\', size=%zu)." +#define LTFS14005E "Path \'%s\' exists, but is not a directory." +#define LTFS14011E "Cannot allocate LTFS volume structure." +#define LTFS14029I "Ready to receive file system requests." +#define LTFS14414I " -o atime Update index if only access times have changed" +#define LTFS14441I " -o verbose= Override output verbosity directly (default: %d)" +#define LTFS14091E "Invalid generation for rollback mount is specified. %s." +#define LTFS14034D "FUSE release directory \'%s\'." +#define LTFS14056E "No driver plug-in configured and no default found." +#define LTFS14018E "Cannot get read-only status of the medium." +#define LTFS14063I "Sync type is \"%s\", Sync time is %ld sec." +#define LTFS14404I " -o work_directory= LTFS work directory (default: %s)" +#define LTFS14042D "FUSE truncate \'%s\' (length=%lld)." +#define LTFS14022E "Cannot flush file data to the medium. Data might be lost (%s)." +#define LTFS14053D "FUSE removexattr \'%s\' (name=\'%s\')." +#define LTFS14023E "Cannot set extended attribute: position option must be zero." +#define LTFS14111I "Initial setup completed successfully." +#define LTFS14073I "Index partition has no space to write index. Mounting as read-only." +#define LTFS14427I " -o sync_type= Specify sync type (default: time@5)\n should be specified as follows:\n time@min: LTFS attempts to write an index each 'min' minutes.\n min should be a decimal number from 1 to %ld\n" +#define LTFS14417I " -o umask= Override default permission mask (3 octal digits, default: 000)" +#define LTFS14002E "Cannot set up permissions (%d)." +#define LTFS14090E "KMI backend option parsing failed." +#define LTFS14467I " -o syslogtrace Enable diagnostic output to stderr and syslog(same as verbose=303)" +#define LTFS14437I " -o rollback_mount= Attempt to mount on previous index generation (read-only mount)" +#define LTFS14116E "This medium is not supported (%d)." +#define LTFS14027E "Cannot read directory: convert to system locale failed (%s, %d)." +#define LTFS14037D "FUSE flush \'%s\'." +#define LTFS14047D "FUSE readdir \'%s\'." +#define LTFS14060E "Failed to convert the sync time (%s)." +#define LTFS14076I "Attempting to mount the cartridge without EOD existence check." +#define LTFS14448I " -o release_device Clear device reservation (should be specified with -o devname" +#define LTFS14007E "Invalid fmask \'%s\' (must be 3 octal digits, such as 022)." +#define LTFS14412I " -o gid=N Set file group" +#define LTFS14017E "Cannot parse data placement rules (%d)." +#define LTFS14408I " -h, --help Display this help and exit" +#define LTFS14400I "usage: %s mountpoint [options]" +#define LTFS14424I " -o fulltrace Enable full call tracing (same as verbose=4)" +#define LTFS14406I " -a Advanced help, including standard FUSE options" +#define LTFS14036D "FUSE fsync \'%s\'." +#define LTFS14105I "%s." +#define LTFS14455I " -o kmi_backend= Key manager interface implementation to use (default: %s, use \"none\" to disable)" +#define LTFS14044D "FUSE unlink \'%s\'." +#define LTFS14075E "Cannot set up tape drive." +#define LTFS14004E "Cannot create work directory (%d)." +#define LTFS14051D "FUSE getxattr \'%s\' (name=\'%s\')." +#define LTFS14415I " -o tape_backend= tape backend to use (default: %s)" +#define LTFS14016E "Cannot format data placement rules (%d)." +#define LTFS14409I "FUSE options:" +#define LTFS14440I " -o noatime Do not update index if only access times have changed (default)" +#define LTFS14439I " -o noeject Do not eject the cartridge after unmount (default)" +#define LTFS14401I "LTFS options:" +#define LTFS14068E "Specified sync time is too big or too small (%ld)." +#define LTFS14028W "Cannot initialize I/O scheduler. The scheduler is disabled. Performance might decrease and memory usage might increase." +#define LTFS14425I " -o eject Eject the cartridge after unmount" +#define LTFS31395D "Backend %s: %zu bytes %s" +#define LTFS31287I "Taking drive dump in buffer" +#define LTFS31282D "Transferring dump data" +#define LTFS31284W "Cannot retrieve drive dump: failed to write to dump file (%d)" +#define LTFS31280D "Total dump data length is %lld." +#define LTFS31252I "A long data wipe is in progress. %d %%" +#define LTFS31278I "Saving drive dump to %s" +#define LTFS31253W "Failed to get medium type code: medium type check is skipped" +#define LTFS31246W "Received low space warning (programmable early warning) in %s" +#define LTFS31285D "Transfer %d: wrote %d bytes" +#define LTFS31268I "Cannot allocate memory in %s" +#define LTFS31248D "EOD detected (%s): ignore sense" +#define LTFS31262I "Cannot parse the log page 0x%02x in %s" +#define LTFS31219I "Option parsing for the CAM backend failed (%d)" +#define LTFS31272I "Logical block protection is disabled" +#define LTFS31260I "Cannot read attribute (%d)" +#define LTFS31235I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." +#define LTFS31399I "CAM backend options:\n" +#define LTFS31209D "IOCTL: no sense info" +#define LTFS31225I "Cannot open device \'%s\' (%d)" +#define LTFS31286W "Cannot retrieve drive dump: wrote %d bytes out, expected %d" +#define LTFS31237I "Read block: overlength condition is detected. residual = %d, actual = %d" +#define LTFS31270E "Encryption feature is not supported on the drive: %d" +#define LTFS31259I "Cannot get remaining capacity: loop index error (%d)" +#define LTFS31255I "Unsupported cartridge (0x%x, 0x%x)" +#define LTFS31264W "Cannot get EOD status: failed to get log page 0x%02x (%d)" +#define LTFS31223I "Opening a device through CAM driver (%s)" +#define LTFS31257I "Cannot get remaining capacity: get log page 0x%02x failed (%d)" +#define LTFS31397D "Backend %s: (%llu, %llu) %s" +#define LTFS31393D "Backend %s: %d %s" +#define LTFS31207D "IOCTL: pos = 0x%02x%02x%02x%02x %s" +#define LTFS31211D "IOCTL: %s %d expected error %d. retry the operation." +#define LTFS31229I "Vendor ID is %s" +#define LTFS31239E "Logical block protection Error on read" +#define LTFS31251I "A long data wipe is in progress. (%d minutes passed)" +#define LTFS31213I "Error on %s: %s (%d) %s" +#define LTFS31269E "Encryption method of the drive is not AME but %s (0x%02X)" +#define LTFS31265W "Cannot get EOD status: failed to parse the log page" +#define LTFS31226W "Device \'%s\' must be opened in read-only mode" +#define LTFS31281D "Total number of transfers is %d." +#define LTFS31267I "Unexpected parameter size for getting active CQ loss write (%d)" +#define LTFS31232I "Firmware revision is %s" +#define LTFS31236D "Read block: file mark detected" +#define LTFS31283W "Cannot retrieve drive dump: failed to read buffer (%d)" +#define LTFS31224E "%s: medium is already mounted or in use" +#define LTFS31261I "Cannot get log page 0x%02x (%d) in %s" +#define LTFS31234I "Pseudo-error on %s" +#define LTFS31279W "Cannot retrieve drive dump: failed to create dump file (%d)" +#define LTFS31247E "Logical block protection Error on write" +#define LTFS31208I "IOCTL: %s %d returns %d (generic %d) %s" +#define LTFS31249I "Unrecognized space type" +#define LTFS31250I "Cannot space: count value %d of the space command is too large" +#define LTFS31220E "Invalid scsi_lbprotect option: %s" +#define LTFS31288I "Forcing drive dump" +#define LTFS31218I "Parsing log page: buffer too small, copying %zu bytes from %lx" +#define LTFS31263E "Failed to open file '%s' (%d)" +#define LTFS31266W "Cannot get EOD status: value length or partition number is wrong %d - (%d, %d)" +#define LTFS31258I "Cannot get remaining capacity: failed to parse the log page" +#define LTFS31271I "Logical block protection is enabled" +#define LTFS31214E "Error on %s: msg = NULL (%d) %s" +#define LTFS31230I "Unsupported Drive \'%s\'" +#define LTFS31392D "Backend %s %s" +#define LTFS31398D "Backend %s: (%llu, %llu) FM = %llu %s" +#define LTFS31289I "Failed to get cartridge status. The cartridge is not loaded." +#define LTFS31245W "Received low space warning (early warning) in %s" +#define LTFS31394D "Backend %s: %llu %s" +#define LTFS31212I "Cannot get sense (%d)" +#define LTFS31233I "Drive serial is %s" +#define LTFS31206D "IOCTL: sense = %02x/%02x%02x" +#define LTFS31396D "Backend %s: %zu %s" +#define LTFS31228I "Product ID is \'%s\'" +#define LTFS31256I "Invalid format mode (%d)" +#define LTFS31238D "Read block: underlength condition is detected. shortage = %d, actual = %d" +#define LTFS13015D "Unified I/O scheduler initialized." +#define LTFS13006E "Cannot initialize scheduler: failed to initialize mutex %s (%d)." +#define LTFS13026E "Write perm handling error : %s (%d)." +#define LTFS13007E "Cannot initialize scheduler: failed to initialize condition variable %s (%d)." +#define LTFS13013W "Index partition writer: failed to write data to the tape (%d)." +#define LTFS13009E "Failed to initialize mutex in scheduler private data (%d)." +#define LTFS13008E "Cannot initialize scheduler: failed to create thread %s (%d)." +#define LTFS13019E "Cannot flush: failed to write to data partition (%d)." +#define LTFS13022W "Freeing a dentry priv with outstanding write requests. This is a bug." +#define LTFS13012E "Invalid request_state received when updating the queue membership (%d)." +#define LTFS13018E "Cannot write: failed to allocate a write request." +#define LTFS13020E "Aborting full flush: flushing dentry '%s' failed (%d)." +#define LTFS13014W "Data partition writer: failed to write data to the tape (%d)." +#define LTFS13011E "Invalid back pointer to the dentry in the dentry_priv structure." +#define LTFS13025I "Get error position (%d, %d)." +#define LTFS13024I "Clean up extents and append index at index partition (%d)." +#define LTFS13021W "Failed to save index partition extents for a dentry (%d)." +#define LTFS13010E "Cannot write: failed to allocate scheduler private data (%d)." +#define LTFS13005E "Cannot initialize scheduler: failed to initialize cache manager." +#define LTFS13016D "Unified I/O scheduler uninitialized." +#define LTFS13017E "Cannot write: failed to allocate a cache block (%d)." +#define LTFS30001I "Opening a redirecting file through generic file driver (%s)." +#define LTFS30015E "Cannot read: no block at current position." +#define LTFS30154W "Cannot parse cartridge configuration file: 0-byte output." +#define LTFS30049E "Cannot allocate memory for cartridge configuration file (%d)." +#define LTFS30031E "Cannot write file marks: failed to remove current record (%d)." +#define LTFS30055E "Cannot set capacity: must issue command from partition 0, block 0." +#define LTFS30066E "Cannot write attribute: failed to write file (%d)." +#define LTFS30009E "Cannot read: check for file mark failed (%d)." +#define LTFS30074E "Cannot update EOD: failed to remove file (%d)." +#define LTFS30025E "Cannot write: failed to write file (%d)." +#define LTFS30041E "Cannot space: Unrecognized space type." +#define LTFS30039E "Cannot locate: failed to generate file name." +#define LTFS30086I "Cartridge is unsupported (%s, 0x%02x)." +#define LTFS30002E "Cannot open redirecting file (%s)." +#define LTFS30014D "Backend read: returning %zd bytes." +#define LTFS30030E "Cannot write file marks: failed to set write pass (%d)." +#define LTFS30067E "Cannot set compression: unit not ready." +#define LTFS30054E "Cannot read position: unit not ready." +#define LTFS30077E "Cannot space file marks: tried to space over EOD." +#define LTFS30155E "Cannot parse cartridge configuration file: toptag = %s." +#define LTFS30150E "Cannot parse cartridge configuration file: start document (%d)." +#define LTFS30022E "Cannot write: failed to set write pass (%d)." +#define LTFS30042E "Cannot space: failed to generate file name." +#define LTFS30198D "Backend %s: (%llu, %llu) FM = %llu." +#define LTFS30038E "Cannot locate: invalid partition %lu." +#define LTFS30050E "Cartridge configuration file is already existed as directory (%d)." +#define LTFS30037E "Cannot locate: unit not ready." +#define LTFS30195D "Backend %s." +#define LTFS30007E "Pseudo-error on %s." +#define LTFS30072E "Cannot update EOD: failed to generate file name." +#define LTFS30017E "Cannot write: unit not ready." +#define LTFS30047E "Cannot find the location pointed by redirecting file (%s)." +#define LTFS30060E "Cannot lock medium: unit not ready." +#define LTFS30057E "Cannot format: unknown format type." +#define LTFS30156E "Cannot parse cartridge configuration file: encoding = %s." +#define LTFS30027E "Cannot write: failed to update EOD (%d)." +#define LTFS30068E "Cannot find EOD: failed to generate file name." +#define LTFS30084D "Found a device (%s, %s, %s, %s)." +#define LTFS30036E "Cannot rewind: unit not ready." +#define LTFS30000I "Opening a device through generic file driver (%s)." +#define LTFS30016D "Backend write: %u bytes (at position=(%u, %llu), FMs %llu)." +#define LTFS30082I "No device directory is specified (%s)." +#define LTFS30048I "Loading a directory through generic file driver (%s)." +#define LTFS30056E "Cannot format: must issue command from partition 0, block 0." +#define LTFS30061E "Cannot unlock medium: unit not ready." +#define LTFS30157E "Cannot create xml reader for cartridge configuration file." +#define LTFS30008E "Cannot read: already at EOD." +#define LTFS30024E "Cannot write: failed to create file \'%s\' (%d)." +#define LTFS30075E "Cannot remove record: failed to make file name." +#define LTFS30040E "Cannot space: unit not ready." +#define LTFS30085I "Cartridge is read-only (%d, %s)." +#define LTFS30069E "Cannot find EOD: failed to check for file (%d)." +#define LTFS30011E "Cannot read: failed to open file (%d)." +#define LTFS30029E "Cannot write file marks: unit not ready." +#define LTFS30197D "Backend %s: (%llu, %llu)." +#define LTFS30035E "Cannot write file marks: failed to update EOD (%d)." +#define LTFS30059E "Device already reserved." +#define LTFS30005D "Backend read: %u bytes (from position=(%u, %llu), FMs %llu)." +#define LTFS30019I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." +#define LTFS30070E "Cannot find EOD: failed to update EOD (%d)." +#define LTFS30021E "Cannot write: failed to remove current record (%d)." +#define LTFS30045E "Cannot get a redirect location (%s)." +#define LTFS30051E "Cannot load cartridge configuration file (%d)." +#define LTFS30158E "Cannot parse cartridge configuration file: XML." +#define LTFS30078E "Cannot space file marks: failed to generate file name." +#define LTFS30062W "Cannot read attribute: failed to open file (%d)." +#define LTFS30010E "Cannot read: check for record failed (%d)." +#define LTFS30034E "Cannot write file marks: failed to close file (%d)." +#define LTFS30006E "Cannot read: unit not ready." +#define LTFS30058E "Cannot get remaining capacity: unit not ready." +#define LTFS30073E "Cannot update EOD: failed to create file (%d)." +#define LTFS30151E "Cannot parse cartridge configuration file: end document (%d)." +#define LTFS30046I "Dummy device shows empty (%s)." +#define LTFS30018E "Cannot write: null input buffer." +#define LTFS30083I "Scanning the devices directory (%s)." +#define LTFS30026E "Cannot write: failed to close file (%d)." +#define LTFS30063W "Cannot read attribute: failed to read file (%d)." +#define LTFS30013E "Cannot read: failed to close file (%d)." +#define LTFS30033E "Cannot write file marks: failed to create file \'%s\' (%d)." +#define LTFS30003I "Opening a directory through generic file driver (%s)." +#define LTFS30064E "Cannot write attribute: failed to generate file name." +#define LTFS30088I "Unsupported density code (0x%x, 0x%02x)." +#define LTFS30152E "Cannot parse cartridge configuration file: writer creation." +#define LTFS30023E "Cannot write: failed to generate file name." +#define LTFS30076E "Cannot remove record: failed to unlink file (%d)." +#define LTFS30043E "Cannot erase: unit not ready." +#define LTFS30199I "FILE backend options:\n -o devname= LTFS emulation directory (default=%s)\n" +#define LTFS30053E "Cannot load: failed to get write pass." +#define LTFS30080E "Cannot space file marks: tried to space over BOT." +#define LTFS30028D "Backend write file marks: %u (at position=(%u, %llu), FMs %llu)." +#define LTFS30012E "Cannot read: failed to read file (%d)." +#define LTFS30196D "Backend %s: %llu." +#define LTFS30032E "Cannot write file marks: failed to generate file name." +#define LTFS30065E "Cannot write attribute: failed to create file (%d)." +#define LTFS30153E "Cannot parse cartridge configuration file: writer fail." +#define LTFS30004E "Cannot open the device directory (%s)." +#define LTFS30020E "Cannot write: requested size (%u) exceeds maximum block size (%u)." +#define LTFS30071E "Cannot update EOD: failed to remove current record (%d)." +#define LTFS30081I "Getting the device directory (%s)." +#define LTFS30044D "Erase partition %lu." +#define LTFS30052E "Cannot load: failed to find EOD on partition %u (%d)." +#define LTFS30079E "Cannot space file marks: failed to check for file (%d)." +#define LTFS30840W "Cannot get EOD status: failed to get log page 0x%02x (%d)." +#define LTFS30812I "Cannot get the drive identifier of %s." +#define LTFS30821D "Read block: underrun in illegal length. residual = %d, actual = %d." +#define LTFS30841W "Cannot get EOD status: failed to parse the log page." +#define LTFS30802I "Cannot set timeout, The opcode = %02x (%d)." +#define LTFS30845E "Encryption feature is not supported on the drive: %d." +#define LTFS30810I "Opening a device through iokit driver (%s)." +#define LTFS30823I "Drive cleaning required." +#define LTFS30857D "Total dump data length is %lld." +#define LTFS30855I "Saving drive dump to %s." +#define LTFS30838I "Cannot parse the log page 0x%02x in %s." +#define LTFS30843E "Invalid scsi_lbprotect option: %s." +#define LTFS30827D "EOD detected (%s): ignore sense." +#define LTFS30805D "CDB %s." +#define LTFS30816I "Firmware revision is %s." +#define LTFS30829I "A long data wipe is in progress. (%d minutes passed)." +#define LTFS30806I "Unexpected CDB status: 0x%08x." +#define LTFS30826W "Received low space warning (programmable early warning) in %s." +#define LTFS30853I "Logical block protection is enabled." +#define LTFS30814I "Vendor ID is %s." +#define LTFS30825W "Received low space warning (early warning) in %s." +#define LTFS30800I "Cannot set scatter gather, The opcode = %02x (%d)." +#define LTFS30804D "CDB check condition: sense = %06x, %s." +#define LTFS30819W "A length mismatch is detected. (Act = %d, resid = %d, resid_sense = %d)." +#define LTFS30832I "Cannot get remaining capacity: get log page 0x%02x failed (%d)." +#define LTFS30837I "Cannot get log page 0x%02x (%d) in %s." +#define LTFS30859D "Transferring dump data." +#define LTFS30836I "Cannot read attribute (%d)." +#define LTFS30834I "Unexpected field in remaining capacity (0x%x)." +#define LTFS30869W "The drive is already reserved: %s (%s)" +#define LTFS30998D "Backend %s: (%llu, %llu) FM = %llu %s." +#define LTFS30867W "WWPID of reservation: x%02x%02x%02x%02x%02x%02x%02x%02x (%s)" +#define LTFS30865I "%s returns %s (%d) %s." +#define LTFS30835I "Cannot get remaining capacity: failed to parse the log page." +#define LTFS30809I "Cannot open device: inquiry failed (%d)." +#define LTFS30833I "Cannot parse remaining capacity (0x%x, %d)." +#define LTFS30844E "Encryption method of the drive is not AME but %s (0x%02X)." +#define LTFS30861W "Cannot retrieve drive dump: failed to write to dump file (%d)." +#define LTFS30868I "Retry to reserve from key registration (%s)" +#define LTFS30801I "Cannot set CDB, The opcode = %02x (%d)." +#define LTFS30999I "iokit backend options:\n" +#define LTFS30811I "Cannot open device: failed to convert devname to devnumber (%s)." +#define LTFS30860W "Cannot retrieve drive dump: failed to read buffer (%d)." +#define LTFS30803I "Failed to execute CDB, The opcode = %02x (%d)." +#define LTFS30824E "Logical block protection Error on read." +#define LTFS30839I "Unexpected parameter size for getting active CQ loss write (%d)." +#define LTFS30822D "Read block: file mark detected." +#define LTFS30820I "Read block: overrun in illegal length. residual = %d, actual = %d." +#define LTFS30846I "Pseudo-error on %s." +#define LTFS30854I "Logical block protection is disabled." +#define LTFS30870I "Failed to get the cartridge status. The cartridge is not loaded." +#define LTFS30817I "Drive serial is %s." +#define LTFS30996D "Backend %s: %llu %s." +#define LTFS30828I "Unrecognized space type." +#define LTFS30807D "%s (0x%02x) expected error %d." +#define LTFS30856W "Cannot retrieve drive dump: failed to create dump file (%d)." +#define LTFS30815I "Product ID is \'%s\'." +#define LTFS30995D "Backend %s: %zu bytes %s." +#define LTFS30847I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." +#define LTFS30997D "Backend %s: (%llu, %llu) %s." +#define LTFS30813I "Unsupported drive \'%s\' / \'%s\'." +#define LTFS30863I "Taking drive dump in buffer." +#define LTFS30994D "Backend %s: %zu %s." +#define LTFS30992D "Backend %s %s." +#define LTFS30866E "%s returns msg = NULL (%d) %s." +#define LTFS30858D "Total number of transfers is %d." +#define LTFS30842W "Cannot get EOD status: value length or partition number is wrong %d - (%d, %d)." +#define LTFS30831I "Unsupported cartridge (0x%x, 0x%x)." +#define LTFS30993D "Backend %s: %d %s." +#define LTFS30808I "%s (0x%02x) returns %d." +#define LTFS30830I "A long data wipe is in progress. %d %%." +#define LTFS30864I "Forcing drive dump." +#define LTFS30862W "Cannot retrieve drive dump: wrote %d bytes out, expected %d." +#define LTFS30818I "Reopening a device through iokit driver (%s)." +#define LTFS30598D "Backend %s: (%llu, %llu) FM = %llu %s." +#define LTFS30485D "Transfer %d: wrote %d bytes." +#define LTFS30442I "Skip retrying write() call at (%u, %llu), because the record was already written (%u, %llu)." +#define LTFS30439E "Logical block protection Error on read." +#define LTFS30478I "Saving drive dump to %s." +#define LTFS30401D "SIOC_PASS_THROUGH: no sense info: T%02x:M%02x:H%02x:D%02x %s." +#define LTFS30415W "Cannot detect lin_tape version." +#define LTFS30425I "Cannot open device \'%s\' (%d)." +#define LTFS30470E "Encryption feature is not supported on the drive: %d." +#define LTFS30465W "Cannot get EOD status: failed to parse the log page." +#define LTFS30483W "Cannot retrieve drive dump: failed to read buffer (%d)." +#define LTFS30423I "Opening a device through ibmtape driver (%s)." +#define LTFS30486W "Cannot retrieve drive dump: wrote %d bytes out, expected %d." +#define LTFS30599I "IBMTAPE backend options:\n -o devname= tape device (default=%s)\n" +#define LTFS30453W "Failed to get medium type code: medium type check is skipped." +#define LTFS30414E "Error on %s: msg = NULL (%d) %s." +#define LTFS30438D "Read block: underlength condition is detected. shortage = %d, actual = %d." +#define LTFS30462I "Cannot parse the log page 0x%02x in %s." +#define LTFS30402D "SIOC_PASS_THROUGH: sense = %02x/%02x%02x." +#define LTFS30430I "Unsupported Drive \'%s\'." +#define LTFS30443W "Enexpected current position (%d). (%u, %llu), (%u, %llu) ." +#define LTFS30408I "IOCTL: %s %d returns %d (generic %d) %s." +#define LTFS30436D "Read block: file mark detected." +#define LTFS30448D "EOD detected (%s): ignore sense." +#define LTFS30440W "Detect ENOMEM in write() call. Wait 3 secs and retry (%d)." +#define LTFS30407D "IOCTL: pos = 0x%02x%02x%02x%02x %s." +#define LTFS30592D "Backend %s %s." +#define LTFS30417E "Old lin_tape version is detected." +#define LTFS30427I "Cannot open device: inquiry failed (%d)." +#define LTFS30456I "Invalid format mode (%d)." +#define LTFS30467I "Unexpected parameter size for getting active CQ loss write (%d)." +#define LTFS30409D "IOCTL: no sense info." +#define LTFS30431I "Cannot open device: inquiry 0x%x failed (%d)." +#define LTFS30449I "Unrecognized space type." +#define LTFS30593D "Backend %s: %d %s." +#define LTFS30441I "Retrying write() call at (%u, %llu)." +#define LTFS30416I "lin_tape version is %s." +#define LTFS30484W "Cannot retrieve drive dump: failed to write to dump file (%d)." +#define LTFS30471I "Logical block protection is enabled." +#define LTFS30479W "Cannot retrieve drive dump: failed to create dump file (%d)." +#define LTFS30455I "Unsupported cartridge (0x%x, 0x%x)." +#define LTFS30400I "SIOC_PASS_THROUGH: command failed (%d) %s." +#define LTFS30464W "Cannot get EOD status: failed to get log page 0x%02x (%d)." +#define LTFS30434I "Pseudo-error on %s." +#define LTFS30418I "Parsing log page: buffer too small, copying %zu bytes from %lx." +#define LTFS30426W "Device \'%s\' must be opened in read-only mode." +#define LTFS30489I "Failed to get cartridge status. The cartridge is not loaded." +#define LTFS30458I "Cannot get remaining capacity: failed to parse the log page." +#define LTFS30429I "Vendor ID is %s." +#define LTFS30594D "Backend %s: %llu %s." +#define LTFS30446W "Received low space warning (programmable early warning) in %s." +#define LTFS30450I "Cannot space: count value %d of the space command is too large." +#define LTFS30481D "Total number of transfers is %d." +#define LTFS30405I "SIOC_PASS_THROUGH: status info: T%02x:M%02x:H%02x:D%02x %s." +#define LTFS30411D "IOCTL: %s %d expected error %d. retry the operation." +#define LTFS30461I "Cannot get log page 0x%02x (%d) in %s." +#define LTFS30469E "Encryption method of the drive is not AME but %s (0x%02X)." +#define LTFS30437I "Read block: overlength condition is detected. residual = %d, actual = %d." +#define LTFS30406D "IOCTL: sense = %02x/%02x%02x." +#define LTFS30466W "Cannot get EOD status: value length or partition number is wrong %d - (%d, %d)." +#define LTFS30482D "Transferring dump data." +#define LTFS30595D "Backend %s: %zu bytes %s." +#define LTFS30410D "IOCTL: %s %d expected error %d." +#define LTFS30457I "Cannot get remaining capacity: get log page 0x%02x failed (%d)." +#define LTFS30447E "Logical block protection Error on write." +#define LTFS30420E "Invalid scsi_lbprotect option: %s." +#define LTFS30487I "Taking drive dump in buffer." +#define LTFS30452I "A long data wipe is in progress. %d %%." +#define LTFS30463E "Failed to open file '%s' (%d)." +#define LTFS30444W "Position confirmation for write() call retry is failed (%d). (%u, %llu), (%u, %llu)." +#define LTFS30403D "SIOC_PASS_THROUGH: pos = 0x%02x%02x%02x%02x %s." +#define LTFS30596D "Backend %s: %zu %s." +#define LTFS30433I "Drive serial is %s." +#define LTFS30413I "Error on %s: %s (%d) %s." +#define LTFS30472I "Logical block protection is disabled." +#define LTFS30435I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." +#define LTFS30419I "Option parsing for the ibmtape backend failed (%d)." +#define LTFS30424E "%s: medium is already mounted or in use." +#define LTFS30488I "Forcing drive dump." +#define LTFS30459I "Cannot get remaining capacity: loop index error (%d)." +#define LTFS30428I "Product ID is \'%s\'." +#define LTFS30451I "A long data wipe is in progress. (%d minutes passed)." +#define LTFS30404I "SIOC_PASS_THROUGH: cannot sense data from the drive." +#define LTFS30597D "Backend %s: (%llu, %llu) %s." +#define LTFS30445W "Received low space warning (early warning) in %s." +#define LTFS30460I "Cannot read attribute (%d)." +#define LTFS30480D "Total dump data length is %lld." +#define LTFS30468I "Cannot allocate memory in %s." +#define LTFS30432I "Firmware revision is %s." +#define LTFS30412I "Cannot get sense (%d)." +#define LTFS39810W "No IP address was found. Use host name based reservation key." +#define LTFS39805W "The timeout table is not configured: SCSI OP code 0x%02x. Use the default timeout." +#define LTFS39801D "SCSI timeout (op_code 0x%02x, timeout = %d)." +#define LTFS39811W "Cannot fetch network I/F information. Use host name based reservation key. (%d)" +#define LTFS39809D "WORM cartridge is loaded." +#define LTFS39803E "CRC check failed: Len = %d, Actual CRC = %08x, Expected CRC = %08x." +#define LTFS39804D "CRC: %s ,Len = %d, CRC = %08x." +#define LTFS39813W "Drive firmware level does not correctly detect the EOD status." +#define LTFS39806D "Is medium mountable: six-character bar code %s." +#define LTFS39800W "Unsupported SCSI OP code 0x%02x." +#define LTFS39812W "Drive firmware must be updated. Upgrading to %s or later is recommended." +#define LTFS39808I "Is medium mountable: unsupported medium %s is detected." +#define LTFS39807E "Is medium mountable: invalid bar code %s." +#define LTFS39802W "Unknown SCSI OP code 0x%02x, use default timeout." +#define LTFS15506E "Failed to parse data key and/or data key identifier." +#define LTFS15508I "Key manager interface simple plug-in options:\n" +#define LTFS15507E "Cannot find data key of the data key identifier." +#define LTFS15505E "Invalid sequence error (%d,%d): %s." +#define LTFS15504E "Option parsing for the key manager interface backend failed (%d)." +#define LTFS15502E "Encryption key format violation (%s): %s." +#define LTFS15503E "Cannot find data key." +#define LTFS15501D "Simple plug-in uninitialized." +#define LTFS15500D "Simple plug-in initialized." +#define LTFS15413I " -i, --config= Use the specified configuration file (default: %s)" +#define LTFS15023I "Formatting failed." +#define LTFS15057I "Need to specify the correct data key or -f option to format this medium." +#define LTFS15012E "Failed to format the medium." +#define LTFS15055E "Unknown option '%s %s'." +#define LTFS15031E "Volume name must conform to file name rules." +#define LTFS15062E "Failed to unformat the medium due to WORM error." +#define LTFS15053E "Key manager interface backend option parsing failed." +#define LTFS15004I "LTFS volume blocksize: %lu." +#define LTFS15049I "Checking the medium (%s)." +#define LTFS15424I " --long-wipe Unformat the medium and erase any data on the tape by overwriting special data pattern.\n This operation takes over 3 hours. Once you start, you cannot interrupt it" +#define LTFS15409I " -h, --help This help" +#define LTFS15417I " -x, --fulltrace Enable full function call tracing (slow)" +#define LTFS15021W "Cannot unload backend." +#define LTFS15410I "Usage example:" +#define LTFS15059I "%s version %s." +#define LTFS15040I "Medium unformatted successfully." +#define LTFS15058E "Cannot set the tape attribute: %s." +#define LTFS15010I "Creating data partition %c on SCSI partition %d." +#define LTFS15401I "Available options are:" +#define LTFS15042I "%s." +#define LTFS15011I "Creating index partition %c on SCSI partition %d." +#define LTFS15421I " --kmi-backend= Use the specified key manager interface backend (default: %s)" +#define LTFS15411I " %s --device=%s --rules=\"%s\"" +#define LTFS15028E "Block size must be at least %d." +#define LTFS15415I " -b, --blocksize= Set the LTFS record size (default: %d)" +#define LTFS15015E "Cannot parse data placement rules (%d)." +#define LTFS15052E "Could not initialize the key manager interface plug-in. \'%s\' (%d)." +#define LTFS15038E "Failed to unformat the medium." +#define LTFS15024I "Medium formatted successfully." +#define LTFS15050E "Cannot open key manager interface backend \'%s\'." +#define LTFS15402I " -d, --device= Tape device (required)" +#define LTFS15001E "Cannot allocate LTFS volume structure." +#define LTFS15423I " -V, --version Version information" +#define LTFS15003I "Formatting device \'%s\'." +#define LTFS15007D "Device opened." +#define LTFS15406I " --no-override Disallow mount-time data placement policy changes" +#define LTFS15047E "Medium is already formatted (%d)." +#define LTFS15034E "Cannot format data placement rules (%d)." +#define LTFS15054E "Tape backend option parsing failed." +#define LTFS15048I "Need to specify -f or --force option to format this medium." +#define LTFS15020D "Closing the device." +#define LTFS15418I " -w, --wipe Restore the LTFS medium to an unpartitioned medium (format to a legacy scratch medium)" +#define LTFS15022D "Device closed." +#define LTFS15404I " -n, --volume-name= Tape volume name (empty by default)" +#define LTFS15005I "Index partition placement policy: %s." +#define LTFS15013I "Volume UUID is: %s." +#define LTFS15030E "Tape serial must contain only ASCII digits and capital letters." +#define LTFS15008E "Cannot open backend \'%s\'." +#define LTFS15009E "Cannot open device \'%s\' (%d)." +#define LTFS15046E "Unformatting was canceled by the user." +#define LTFS15056E "Failed to decrypt the medium." +#define LTFS15025D "Validating command line options." +#define LTFS15414I " -e, --backend= Use the specified tape device backend (default: %s)" +#define LTFS15420I " -f, --force Force to format medium" +#define LTFS15060E "Index criteria update is not allowed on this medium." +#define LTFS15019I "Volume capacity is %llu GB." +#define LTFS15041I "Launched by \"%s\"." +#define LTFS15412I " -p, --advanced-help Full help, including advanced options" +#define LTFS15044E "Cannot set up tape device." +#define LTFS15043I "GCC version is %s." +#define LTFS15061E "Failed to format the medium due to WORM error." +#define LTFS15051W "Cannot unload key manager interface backend." +#define LTFS15045E "Formatting was canceled by the user." +#define LTFS15029E "Tape serial must be 6 characters." +#define LTFS15416I " -c, --no-compression Disable compression on the volume" +#define LTFS15400I "Usage: %s " +#define LTFS15026E "Device name must use the \'%s\' option." +#define LTFS15014E "Cannot set policy override flag in the index (%d)." +#define LTFS15002E "Option validation failed." +#define LTFS15403I " -s, --tape-serial= Tape serial number (6 alphanumeric ASCII characters)" +#define LTFS15405I " -r, --rules= Rules for choosing files to write to the index partition.\n The syntax of the rule argument is:\n size=1M\n size=1M/name=pattern\n size=1M/name=pattern1:pattern2:pattern3\n A file is written to the index partition if it is no larger\n than the given size AND matches at least one of the name\n patterns (if specified). The size argument accepts K, M, and G\n suffixes. Name patterns might contain the special characters\n '?' (match any single character) and '*' (match zero or more\n characters)" +#define LTFS15422I " --syslogtrace Enable diagnostic output to stderr and syslog" +#define LTFS15000I "Starting mkltfs, %s version %s, log level %d." +#define LTFS15037D "Command line options are valid." +#define LTFS15419I " -k, --keep-capacity Keep the tape medium's total capacity proportion" +#define LTFS15407I " -q, --quiet Suppress progress information and general messages" +#define LTFS15039I "Unformatting failed." +#define LTFS15006D "Opening the device." +#define LTFS15408I " -t, --trace Enable function call tracing" +#define LTFS15562E "Encryption key format violation (%s): %s." +#define LTFS15563E "Cannot find data key." +#define LTFS15551D "Flat File plug-in uninitialized." +#define LTFS15550D "Flat File plug-in initialized." +#define LTFS15564E "Option parsing for the key manager interface backend failed (%d)." +#define LTFS15552E "Flat File plug-in failed to parse options." +#define LTFS15565E "Invalid sequence error (%d,%d): %s." +#define LTFS15567E "Cannot find data key of the data key identifier." +#define LTFS15553E "Flat File plug-in failed to open the file (%s, %d)." +#define LTFS15568I "Key manager interface flatfile plug-in options:\n" +#define LTFS15566E "Failed to parse data key and/or data key identifier." +#define LTFS15554E "Flat File plug-in detected key format violation." diff --git a/src/tape_drivers/linux/sg/quantum_tape.c b/src/tape_drivers/linux/sg/quantum_tape.c new file mode 120000 index 00000000..38c99925 --- /dev/null +++ b/src/tape_drivers/linux/sg/quantum_tape.c @@ -0,0 +1 @@ +../../quantum_tape.c \ No newline at end of file diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index 8b2a958b..e72eb2db 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -1268,7 +1268,7 @@ int sg_open(const char *devname, void **handle) ret = DEVICE_GOOD; break; } else if (ret < 0) { - ltfsmsg(LTFS_INFO, 30289I, priv->devname); + ltfsmsg(LTFS_INFO, 30289I, priv->devname, ret); close(priv->dev.fd); priv->dev.fd = -1; free(priv->devname); @@ -1316,7 +1316,7 @@ int sg_open(const char *devname, void **handle) ioctl(priv->dev.fd, SG_SET_RESERVED_SIZE, &reserved_size); ret = ioctl(priv->dev.fd, SG_GET_RESERVED_SIZE, &reserved_size); if (ret < 0) { - ltfsmsg(LTFS_ERR, 30284E, devname); + ltfsmsg(LTFS_INFO, 30284I, devname); goto free; } ltfsmsg(LTFS_INFO, 30285I, devname, reserved_size); From be1b873c318a56756ed4e6cf7ba138b9bf3d0414 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 1 Dec 2020 10:03:04 +0000 Subject: [PATCH 011/121] Fix build break in FreeBSD --- src/libltfs/fs.c | 2 +- src/tape_drivers/freebsd/cam/Makefile.am | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libltfs/fs.c b/src/libltfs/fs.c index 99c83547..ab95a54f 100644 --- a/src/libltfs/fs.c +++ b/src/libltfs/fs.c @@ -263,7 +263,7 @@ struct dentry * fs_allocate_dentry(struct dentry *parent, const char *name, cons d->platform_safe_name = NULL; } else if (name && !platform_safe_name) { d->name.name = strdup(name); - update_platform_safe_name(d, FALSE, idx); + update_platform_safe_name(d, false, idx); if (! d->name.name || ! d->platform_safe_name) { ltfsmsg(LTFS_ERR, 10001E, "fs_allocate_dentry: name"); if (d->name.name) diff --git a/src/tape_drivers/freebsd/cam/Makefile.am b/src/tape_drivers/freebsd/cam/Makefile.am index bea3527e..6127f989 100644 --- a/src/tape_drivers/freebsd/cam/Makefile.am +++ b/src/tape_drivers/freebsd/cam/Makefile.am @@ -41,8 +41,8 @@ BASENAMES = libtape-cam AM_LIBTOOLFLAGS = --tag=disable-static libtape_cam_la_SOURCES = cam_cmn.c cam_tc.c vendor_compat.c ibm_tape.c hp_tape.c quantum_tape.c -libtape_cam_la_DEPENDENCIES = ../../../../messages/libtape_freebsd_cam_dat.a ../../../libltfs/libltfs.la libtape_cam_la-reed_solomon_crc.lo libtape_cam_la-crc32c_crc.lo libtape_cam_la-ibm_tape.lo -libtape_cam_la_LIBADD = ../../../libltfs/libltfs.la ./libtape_cam_la-reed_solomon_crc.lo ./libtape_cam_la-crc32c_crc.lo ./libtape_cam_la-ibm_tape.lo +libtape_cam_la_DEPENDENCIES = ../../../../messages/libtape_freebsd_cam_dat.a ../../../libltfs/libltfs.la libtape_cam_la-reed_solomon_crc.lo libtape_cam_la-crc32c_crc.lo +libtape_cam_la_LIBADD = ../../../libltfs/libltfs.la ./libtape_cam_la-reed_solomon_crc.lo ./libtape_cam_la-crc32c_crc.lo libtape_cam_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../../../messages -ltape_freebsd_cam_dat libtape_cam_la_CPPFLAGS = @AM_CPPFLAGS@ -I ../../.. -I ../.. From fffa1259ed18f90002da1bdb2fd607144d7064be Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 9 Dec 2020 16:13:14 +0900 Subject: [PATCH 012/121] Handle write-perm on LOCATE correctly (#233) --- messages/internal_error/root.txt | 1 + messages/libltfs/root.txt | 1 + src/libltfs/arch/errormap.c | 1 + src/libltfs/ltfs_error.h | 1 + src/libltfs/tape.c | 29 ++++++++++++++++++++++++++--- 5 files changed, 30 insertions(+), 3 deletions(-) diff --git a/messages/internal_error/root.txt b/messages/internal_error/root.txt index 685b26dc..4ca96eec 100644 --- a/messages/internal_error/root.txt +++ b/messages/internal_error/root.txt @@ -251,6 +251,7 @@ root:table { I1198E:string{ "The tape is already removed." } I1199E:string{ "You need to move tape to a storage slot first before the operation." } I1200E:string{ "The operation needs to be started again." } + I1201E:string{ "Locate returns write-perm error." } // Special error codes I9997E:string{ "Child process error (ltfsck/mkltfs): %s (%d)." } diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index eb867875..dfe74acc 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -808,6 +808,7 @@ v 17264I:string { "The index on %s is newer, but MAM shows a permanent write error happened on %s." } 17265I:string { "Skip writing the index because of %s." } 17266I:string { "Skip setting the append only mode because the drive doesn't seem to support it." } + 17267E:string { "Locate command returns write-perm error (%d). Replace a return code to %d." } // For Debug 19999I:string { "%s %s %d." } diff --git a/src/libltfs/arch/errormap.c b/src/libltfs/arch/errormap.c index 748f1bd1..f3ad3d52 100644 --- a/src/libltfs/arch/errormap.c +++ b/src/libltfs/arch/errormap.c @@ -286,6 +286,7 @@ static struct error_map fuse_error_list[] = { { LTFS_TAPE_REMOVED, "I1198E", EIDRM}, { LTFS_NEED_MOVE, "I1199E", EINVAL}, { LTFS_NEED_START_OVER, "I1200E", EINVAL}, + { LTFS_LOCATE_ERROR, "I1201E", EIO}, { EDEV_NO_SENSE, "D0000E", EIO}, { EDEV_OVERRUN, "D0002E", EIO}, diff --git a/src/libltfs/ltfs_error.h b/src/libltfs/ltfs_error.h index a7abe8ce..d7aa6034 100644 --- a/src/libltfs/ltfs_error.h +++ b/src/libltfs/ltfs_error.h @@ -257,6 +257,7 @@ #define LTFS_TAPE_REMOVED 1198 /* The tape is already removed */ #define LTFS_NEED_MOVE 1199 /* Need to move tape to a storage slot first before the operation */ #define LTFS_NEED_START_OVER 1200 /* Need operation needs to be started again */ +#define LTFS_LOCATE_ERROR 1201 /* Locate returns write-perm error */ #define LTFS_ERR_MAX 19999 diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index f23c9f2a..d6ae97c9 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -851,6 +851,7 @@ int tape_seek_append_position(struct device_data *dev, tape_partition_t prt, boo if (ret < 0) { ltfsmsg(LTFS_ERR, 12033E, ret); dev->write_error = true; + return ret; } @@ -996,7 +997,7 @@ int tape_rewind(struct device_data *dev) */ int tape_seek(struct device_data *dev, struct tc_position *pos) { - int ret; + int ret = 0; CHECK_ARG_NULL(dev, -LTFS_NULL_ARG); CHECK_ARG_NULL(pos, -LTFS_NULL_ARG); @@ -1016,8 +1017,14 @@ int tape_seek(struct device_data *dev, struct tc_position *pos) ltfs_mutex_unlock(&dev->read_only_flag_mutex); } } - else { - ret = 0; + + if (IS_WRITE_PERM(-ret)) { + /* + * LOCATE command must not return a WRITE_PERM related error. + * LOCATE is actually read operation, it doesn't make sense to return a WRITE_PERM at all. + */ + ltfsmsg(LTFS_ERR, 17267E, ret, -LTFS_LOCATE_ERROR); + ret = -LTFS_LOCATE_ERROR; } if (ret == 0 && (dev->position.partition != pos->partition || @@ -1051,6 +1058,14 @@ int tape_seek_eod(struct device_data *dev, tape_partition_t partition) ret = dev->backend->locate(dev->backend_data, seekpos, &dev->position); if (ret < 0) { ltfsmsg(LTFS_ERR, 12039E, ret); + if (IS_WRITE_PERM(-ret)) { + /* + * LOCATE command must not return a WRITE_PERM related error. + * LOCATE is actually read operation, it doesn't make sense to return a WRITE_PERM at all. + */ + ltfsmsg(LTFS_ERR, 17267E, ret, -LTFS_LOCATE_ERROR); + ret = -LTFS_LOCATE_ERROR; + } return ret; } @@ -1547,6 +1562,14 @@ int tape_unformat(struct device_data *dev) ret = dev->backend->locate(dev->backend_data, bom, &dev->position); if (ret < 0) { ltfsmsg(LTFS_ERR, 12054E, ret); + if (IS_WRITE_PERM(-ret)) { + /* + * LOCATE command must not return a WRITE_PERM related error. + * LOCATE is actually read operation, it doesn't make sense to return a WRITE_PERM at all. + */ + ltfsmsg(LTFS_ERR, 17267E, ret, -LTFS_LOCATE_ERROR); + ret = -LTFS_LOCATE_ERROR; + } return ret; } From 58cacdd76a798dfdd3045faf4e8300ce650c1ce2 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 18 Jan 2021 12:58:29 +0900 Subject: [PATCH 013/121] Fix check build (#238) - Use Debian10 on check build correctly (#235) - Fix build checker break on macOS - Remove Ubuntu19(Eoan) because it is no more supported - Add a description to refresh shared library cache by ldconfig -v --- .github/workflows/build-debian10.yml | 2 +- .github/workflows/build-ubuntu-eoan.yml | 17 ----------------- .travis.yml | 4 ++-- README.md | 9 +++++---- 4 files changed, 8 insertions(+), 24 deletions(-) delete mode 100644 .github/workflows/build-ubuntu-eoan.yml diff --git a/.github/workflows/build-debian10.yml b/.github/workflows/build-debian10.yml index b2a0a888..3ff0e4b7 100644 --- a/.github/workflows/build-debian10.yml +++ b/.github/workflows/build-debian10.yml @@ -12,6 +12,6 @@ jobs: uses: actions/checkout@v1 - name: Build LTFS id: build - uses: LinearTapeFileSystem/Debian10-Build@v1.0 + uses: LinearTapeFileSystem/Debian10-Build@v1.1 with: destination: '/tmp/ltfs' diff --git a/.github/workflows/build-ubuntu-eoan.yml b/.github/workflows/build-ubuntu-eoan.yml deleted file mode 100644 index 40f456b0..00000000 --- a/.github/workflows/build-ubuntu-eoan.yml +++ /dev/null @@ -1,17 +0,0 @@ -name: Ubuntu 19.10 Build Job - -on: [push, pull_request] - -jobs: - build: - name: Build on Ubuntu Eoan - runs-on: ubuntu-latest - - steps: - - name: Set up Git repository - uses: actions/checkout@v1 - - name: Build LTFS - id: build - uses: LinearTapeFileSystem/Ubuntu1910-Build@v1.0 - with: - destination: '/tmp/ltfs' diff --git a/.travis.yml b/.travis.yml index ac55a2f1..1a78dc40 100644 --- a/.travis.yml +++ b/.travis.yml @@ -33,9 +33,9 @@ before_install: - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export PATH="$PATH:$ICU_PATH:$LIBXML2_PATH" ; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew update ; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install ossp-uuid gnu-sed ; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew cask install osxfuse ; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install --cask osxfuse ; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew list ; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew cask list ; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew list --cask ; fi - if [[ "$TRAVIS_DIST" == "xenial" ]]; then sudo apt-get update ; fi - if [[ "$TRAVIS_DIST" == "xenial" ]]; then sudo apt-get install -y libfuse2 libfuse-dev ; fi - if [[ "$TRAVIS_DIST" == "xenial" ]]; then sudo apt-get install -y libxml2 libxml2-dev ; fi diff --git a/README.md b/README.md index 640f4df3..f8b30346 100644 --- a/README.md +++ b/README.md @@ -119,6 +119,8 @@ make install `./configure --help` shows various options for build and install. +In some systems, you might need `sudo ldconfig -v` after `make install` to load the shared libraries correctly. + #### Parameter settings of the sg driver LTFS uses the sg driver by default. You can improve reliability to change parameters of the sg driver below. @@ -212,16 +214,15 @@ You need to add `--enable-lintape` as an argument of ./configure script if you w | Ubuntu 16.04 LTS | ppc64le | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | | Ubuntu 18.04 LTS | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2018.04%20Build%20Job/badge.svg?branch=master)| | Ubuntu 18.04 LTS | ppc64le | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | - | Ubuntu 19.10 (Need icu-config) | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2019.10%20Build%20Job/badge.svg?branch=master)| | Ubuntu 20.04 LTS (Need icu-config)| x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2020.04%20Build%20Job/badge.svg?branch=master)| | Debian 9 | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian9%20Build%20Job/badge.svg?branch=master)| | Debian 10 (Need icu-config) | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian10%20Build%20Job/badge.svg?branch=master)| - | ArchLinux 2018.08.01 | x86_64 | OK | - | ArchLinux 2018.12.31 (rolling) | x86_64 | OK | + | ArchLinux 2018.08.01 | x86_64 | Not checked automatically | + | ArchLinux 2018.12.31 (rolling) | x86_64 | Not checked automatically| Currently, automatic build checking is working on GitHub Actions and Travis CI. -For Ubuntu19.10, Ubuntu20.04 and Debian10, dummy `icu-config` is needed in the build machine. See Issue [#153](https://github.com/LinearTapeFileSystem/ltfs/issues/153). +For Ubuntu20.04 and Debian10, dummy `icu-config` is needed in the build machine. See Issue [#153](https://github.com/LinearTapeFileSystem/ltfs/issues/153). ### Build and install on OSX (macOS) From dd0887086466d51151030c54b5de11c638729bc5 Mon Sep 17 00:00:00 2001 From: piste2750 <32239919+piste2750@users.noreply.github.com> Date: Mon, 18 Jan 2021 14:14:42 +0900 Subject: [PATCH 014/121] Support HP drives on the CAM backend (#239) - Fix of issue #225 - Remove accidental committed files --- src/.gitignore | 1 + src/ltfsmsg.h | 1502 ---------------------- src/tape_drivers/freebsd/cam/.gitignore | 1 + src/tape_drivers/freebsd/cam/cam_cmn.c | 3 + src/tape_drivers/freebsd/cam/cam_cmn.h | 1 + src/tape_drivers/freebsd/cam/cam_tc.c | 41 +- src/tape_drivers/linux/sg/.gitignore | 1 + src/tape_drivers/linux/sg/quantum_tape.c | 1 - 8 files changed, 34 insertions(+), 1517 deletions(-) delete mode 100644 src/ltfsmsg.h delete mode 120000 src/tape_drivers/linux/sg/quantum_tape.c diff --git a/src/.gitignore b/src/.gitignore index e645f6e0..294b8bd7 100644 --- a/src/.gitignore +++ b/src/.gitignore @@ -1 +1,2 @@ ltfs +ltfsmsg.h diff --git a/src/ltfsmsg.h b/src/ltfsmsg.h deleted file mode 100644 index 06725ac6..00000000 --- a/src/ltfsmsg.h +++ /dev/null @@ -1,1502 +0,0 @@ -#define LTFS16420I " -v, --traverse= Set traverse mode for listing roll back points. Strategy should be forward or backward. (default: backward)" -#define LTFS16088I "Launched by \"%s\"." -#define LTFS16028I "Compression : %s." -#define LTFS16081D "Latest index generation is %d (%c, %llu)." -#define LTFS16004E "Unexpected condition: str_gen is not specified." -#define LTFS16104E "Could not initialize the key manager interface plug-in. \'%s\' (%d)." -#define LTFS16080E "Cannot check volume (%d)." -#define LTFS16096W "Both EODs are missing. Attempt to list index information." -#define LTFS16020W "Failed to close the device (%d)." -#define LTFS16032I "Size threshold : %llu." -#define LTFS16068E "Cannot roll back: found 2 or more target indexes in one partition (%d)." -#define LTFS16052D "Erase history: spacing back and writing a file mark." -#define LTFS16057E "Cannot roll back: the medium is read-only." -#define LTFS16413I " -p, --advanced-help Full help, including advanced options" -#define LTFS16404I " -r, --rollback Roll back to the point specified by -g" -#define LTFS16403I " -g, --generation= Specify the generation to roll back" -#define LTFS16421I " -z, --deep-recovery Recover EOD missing cartridge.\n" -#define LTFS16089I "%s." -#define LTFS16110E "The --salvage-rollback-points option was specified against a normal cartridge." -#define LTFS16029I "Index partition : ID = %c, SCSI Partition = %u." -#define LTFS16005E "Invalid generation number %s." -#define LTFS16024I "Volser (bar code) : %s." -#define LTFS16107E "Unknown option '%s %s'." -#define LTFS16076E "Cannot list rollback points: failed to traverse the data partition (%d)." -#define LTFS16011E "Cannot open device \'%s\'." -#define LTFS16060E "Cannot roll back the index partition: failed to write an index (%d)." -#define LTFS16056E "Cannot roll back the data partition: failed to write an index (%d)." -#define LTFS16025I "Volume UUID : %s." -#define LTFS16019E "Invalid operation mode." -#define LTFS16021E "Volume is inconsistent and was not corrected." -#define LTFS16412I " -h, --help This help" -#define LTFS16059E "Cannot roll back the index partition: failed to erase history (%d)." -#define LTFS16086I "Volume is rolled back successfully." -#define LTFS16062I "Roll back from the data partition." -#define LTFS16051E "Cannot erase history: failed to space forward 1 file mark (%d)." -#define LTFS16090I "GCC version is %s." -#define LTFS16002E "Option validation failed." -#define LTFS16422I " -m, --full-index-info Display full index information (Effective only for -l option)" -#define LTFS16014I "Checking LTFS file system on \'%s\'." -#define LTFS16030I "Data partition : ID = %c, SCSI Partition = %u." -#define LTFS16075E "Cannot list rollback points: failed to traverse the index partition (%d)." -#define LTFS16102E "Cannot open key manager interface backend \'%s\'." -#define LTFS16045D "Erasing history from (%c, %llu)." -#define LTFS16415I " -e, --backend= Override the default tape device backend" -#define LTFS16022I "Volume is consistent." -#define LTFS16408I " -j, --erase-history Erase history at rollback" -#define LTFS16401I "filesys Device file for the tape drive" -#define LTFS16099E "Use the latest version of LTFS software." -#define LTFS16423I " --kmi-backend= Override the default key manager interface backend" -#define LTFS16400I "Usage: %s [options] filesys" -#define LTFS16087E "Volume is inconsistent. Try to recover consistency with ltfsck first." -#define LTFS16058I "Rolling back from the index partition." -#define LTFS16097E "Both EODs are missing. Roll back operation is not permitted." -#define LTFS16063I "Specified rollback point is the current index. The volume is consistent. No operation is required." -#define LTFS16050D "Erase history: spacing to end of index." -#define LTFS16074E "Cannot list rollback points: failed to load the volume (%d)." -#define LTFS16003E "Must provide search criteria using -g." -#define LTFS16105E "Key manager interface backend option parsing failed." -#define LTFS16033I "Name pattern : %s." -#define LTFS16046D "Rolling back %s: (%c, %llu)." -#define LTFS16414I " -i, --config= Use the specified configuration file (default: %s)" -#define LTFS16023I "LTFS volume information:." -#define LTFS16402I "Available options are:" -#define LTFS16098E "Cannot roll back the cartridge: found unsupported index version." -#define LTFS16109E "This operation is not allowed on this medium (%s)." -#define LTFS16416I " -x, --fulltrace Enable full function call tracing (slow)" -#define LTFS16016E "Invalid search mode." -#define LTFS16424I " --capture-index Capture index information to the current directory (-g is effective for this option)" -#define LTFS16092E "Cannot set up tape drive (%s)." -#define LTFS16108I "%s version %s." -#define LTFS16084I "List indexes in backward direction strategy." -#define LTFS16053E "Cannot erase history: failed to space back 1 file mark (%d)." -#define LTFS16073E "Cannot roll back: failed to find indexes." -#define LTFS16100E "Cannot recover the cartridge: found unsupported index version." -#define LTFS16000I "Starting ltfsck, %s version %s, log level %d." -#define LTFS16407I " -l, --list-rollback-points List rollback points" -#define LTFS16409I " -k, --keep-history Keep history at rollback (default)" -#define LTFS16085E "Unexpected traverse strategy." -#define LTFS16091E "Cannot recover missing EOD (%d)." -#define LTFS16425I " --syslogtrace Enable diagnostic output to stderr and syslog" -#define LTFS16015I "Rolling back LTFS file system on \'%s\'." -#define LTFS16001E "Cannot allocate LTFS volume structure." -#define LTFS16072E "Cannot roll back: failed to traverse the data partition (%d)." -#define LTFS16103W "Cannot unload key manager interface backend." -#define LTFS16009E "Must provide device name." -#define LTFS16031I "Data placement policy information:." -#define LTFS16094E "CM in the cartridge might be corrupted. Try to run ltfs with the \"-o force_mount_no_eod\" option." -#define LTFS16426I " -V, --version Version information" -#define LTFS16079E "Cannot roll back: failed to save index partition append position (%d)." -#define LTFS16082I "Saving latest index to data partition to save history." -#define LTFS16106E "Tape backend option parsing failed." -#define LTFS16006I "Rolling back to generation %d." -#define LTFS16010E "Cannot load backend \'%s\' (%d)." -#define LTFS16055E "Cannot roll back the data partition: failed to erase history (%d)." -#define LTFS16071E "Cannot roll back: failed to traverse the index partition (%d)." -#define LTFS16034I "Policy update : %s." -#define LTFS16018I "Listing LTFS file system rollback points on \'%s\'." -#define LTFS16411I " -t, --trace Enable diagnostic output" -#define LTFS16026I "Format time : %04d-%02d-%02d %02d:%02d:%02d.%09ld %s." -#define LTFS16405I " -n, --no-rollback Do not roll back. Verify the point specified by -g (default)" -#define LTFS16427I " --salvage-rollback-points List the rollback points of the cartridge that has no EOD" -#define LTFS16017I "Verify rollback point on \'%s\'." -#define LTFS16083I "List indexes in forward direction strategy." -#define LTFS16093E "Cannot recover the cartridge with ltfsck." -#define LTFS16027I "Block size : %lu." -#define LTFS16054E "Cannot erase history: failed to write a file mark (%d)." -#define LTFS16070E "Cannot roll back: failed to load the volume (%d)." -#define LTFS16111I "The recovery process is skipped because of a locked cartridge (%d)." -#define LTFS16067I "Rolling back based on the following index chain." -#define LTFS16101E "Please use the latest version os LTFS software or --deep-recovery option." -#define LTFS16406I " -f, --full-recovery Recover extra data blocks into directory %s" -#define LTFS16061E "Cannot roll back: invalid partition ID %c." -#define LTFS16410I " -q, --quiet Suppress informational messages" -#define LTFS31008E "Cannot locate: invalid partition %lu." -#define LTFS31018E "Cannot lock medium: unit not ready." -#define LTFS31026I "Getting the device directory (%s)." -#define LTFS31198D "Backend %s: (%llu, %llu) FM = %llu." -#define LTFS31028I "Scanning the devices directory (%s)." -#define LTFS31021E "Cannot erase: unit not ready." -#define LTFS31000I "Opening a device through generic itdtimage driver (%s)." -#define LTFS31012E "Cannot read position: unit not ready." -#define LTFS31002E "Failed to seek to %lld (%s, %lld)." -#define LTFS31010E "Cannot space: Unrecognized space type." -#define LTFS31016E "Cannot get remaining capacity: unit not ready." -#define LTFS31004D "Backend read: %llu bytes (from position=(%u, %llu), FMs %llu)." -#define LTFS31020D "Read attribute: %d %x." -#define LTFS31006E "Cannot rewind: unit not ready." -#define LTFS31014E "Cannot format: must issue command from partition 0, block 0." -#define LTFS31024E "Cannot set compression: unit not ready." -#define LTFS31197D "Backend %s: (%llu, %llu)." -#define LTFS31195D "Backend %s." -#define LTFS31009E "Cannot space: unit not ready." -#define LTFS31019E "Cannot unlock medium: unit not ready." -#define LTFS31007E "Cannot locate: unit not ready." -#define LTFS31030D "Found a device (%s, %s, %s, %s)." -#define LTFS31199I "itdtimg backend options:\n -o devname= tape device (default=%s)\n." -#define LTFS31027I "No device directory is specified (%s)." -#define LTFS31029E "Cannot open the device directory (%s)." -#define LTFS31001E "Failed to open %s: %s (%llu)." -#define LTFS31013E "Cannot set capacity: must issue command from partition 0, block 0." -#define LTFS31011D "state: CurrPosPart=%lld CurrPosBlck=%lld CurrPosFM=%lld Reserved=%d Locked=%d Ready=%d." -#define LTFS31025E "Cannot space %s: tried to space over BOT." -#define LTFS31005E "Cannot read: unit not ready." -#define LTFS31022D "Erase partition %lu." -#define LTFS31017E "Device already reserved." -#define LTFS31015E "Cannot format: unknown format type." -#define LTFS31003I "Closing device through generic itdtimage driver (%s)." -#define LTFS31196D "Backend %s: %llu." -#define LTFS12035E "Cannot rewind medium: backend call failed (%d)." -#define LTFS10006W "Null argument (%s) to %s." -#define LTFS10012E "Failed to register messages with libltfs (%d)." -#define LTFS11210W "Ignoring unexpected file mark in partition %lu." -#define LTFS17123E "Unexpected genaration value (Gen = %d, MAM IP = %d, MAM DP = %d)." -#define LTFS11245E "Cannot convert UTF-16 to UTF-8: failed to fill output buffer (%d)." -#define LTFS17133E "Failed to unload the cartridge." -#define LTFS11299E "Cannot format: unsupported medium." -#define LTFS11338I "Syncing index of %s %s." -#define LTFS11122E "Cannot set extended attribute: failed to look up the xattr (%d)." -#define LTFS11258I "No index found in the data partition." -#define LTFS17067D "Sync is kicked. (%s)." -#define LTFS11285E "Cannot load %s plug-in \'%s\': failed to load the message bundle (%d)." -#define LTFS11189E "Comparing labels: partition map mismatch." -#define LTFS11025D "Volume is consistent." -#define LTFS11217E "Cannot check medium: failed to write a file mark to the data partition (%d)." -#define LTFS11207E "Missing required index partition back pointer." -#define LTFS12012E "Cannot open device: failed backend open call." -#define LTFS12043E "Cannot write block: device is read-only." -#define LTFS17004E "XML parser: tag \'%s\' must not be empty." -#define LTFS12061W "Cannot get VCI data: unexpected Application Specific Info length %d." -#define LTFS11052E "Cannot unlink: path lookup failed (%d)." -#define LTFS11260E "Plug-in \'%s\' was not found in the configuration file." -#define LTFS11235E "Cannot validate name: failed to iterate code point." -#define LTFS11048E "Cannot create: failed to format the path (%d)." -#define LTFS11095E "Cannot format: the medium is write protected." -#define LTFS10020E "Error on %s: %s (%d, %d)." -#define LTFS17097E "XML parser: two extents overlap." -#define LTFS17087I "Kernel version: %s." -#define LTFS11219E "Cannot check medium: pointer verification failed (%d)." -#define LTFS17241W "Failed to open the advisory lock '%s' (%d)." -#define LTFS17255I "Cannot open %s cache for sync (%d)." -#define LTFS11153E "Cannot parse index criteria: failed to parse name rule (%d)." -#define LTFS12056W "Cannot get Volume Change Reference parameter: read attribute failed (%d)." -#define LTFS11195W "This index belongs to a different volume." -#define LTFS17052E "Cannot generate index data (%d) in file \'%s\'." -#define LTFS11000E "Cannot instantiate LTFS volume: failed to allocate device data." -#define LTFS11055E "Cannot rename: failed to format the destination path (%d)." -#define LTFS9012E "Cannot specify \"--quiet\" with \"--trace\" or \"--fulltrace\"." -#define LTFS17057E "Index writer: failed to start the document (%d)." -#define LTFS11251E "Cannot convert UTF-8 to system locale: failed to fill output buffer (%d)." -#define LTFS11345E "Failed to freeze tape %s because of permanent write errors on the IP and DP." -#define LTFS17104E "Cannot get PEWS: Mode Sense for Device Configuration Extension failed (%d)." -#define LTFS17169I "Detected EOD on the missing EOD cartridge." -#define LTFS11187E "Comparing labels: partition IDs must be lowercase ASCII characters." -#define LTFS17018E "XML parser: unsupported encoding \'%s\'." -#define LTFS11165E "Failed to pop a file name from a null stack pointer." -#define LTFS11087E "Cannot read: expected %u bytes from cache, but received %lu bytes." -#define LTFS11197E "Cannot read index: back pointer is corrupt." -#define LTFS11070E "Cannot write blocks: locate append position failed on partition %c." -#define LTFS11317E "Cannot print help message: unknown type: '%s'." -#define LTFS17074I "Device Name = %s, Vendor ID = %s, Product ID = %s, Serial Number = %s, Product Name =%s." -#define LTFS11068E "Cannot write blocks: multiple repetitions specified with an irregular buffer size." -#define LTFS11307W "All unique IDs for volume %s are used. No new files or directories can be created for the volume." -#define LTFS17201I "Starting a long wipe operation. This can take over 3 hours." -#define LTFS17005E "XML parser: extra content in tag \'%s\'." -#define LTFS17156E "Cannot get WRITE MODE: Mode Sense for Device Configuration Extension failed (%d)." -#define LTFS11127E "Cannot get extended attribute: failed to look up the path (%d)." -#define LTFS17160I "Maximum device block size is %d." -#define LTFS11071E "Cannot write blocks: failed to determine medium position (%d)." -#define LTFS11077E "Cannot write: failed to write blocks to the medium (%d)." -#define LTFS11137E "Cannot remove extended attribute: failed to format the name (%d)." -#define LTFS10001E "Memory allocation failed (%s)." -#define LTFS11102E "Cannot format: failed to write ANSI label (%d) on partition %u." -#define LTFS12053E "Cannot format medium: backend call failed (%d)." -#define LTFS17134E "Failed to reload the cartridge." -#define LTFS17142E "Both EODs are missing." -#define LTFS11277W "Cannot find the %s plug-in \'%s\' at \'%s\'." -#define LTFS11180E "Cannot read partition label: failed to space forward over the trailing file mark (%d)." -#define LTFS17219W "%s is out of range in parsing the index (%s)." -#define LTFS17011E "Cannot instantiate an index parser for file \'%s\'." -#define LTFS17119E "Failed to seek to the final index in %s (%d)." -#define LTFS17251I "File %s (%lld) was opened when this index was written." -#define LTFS17197W "Cannot retrieve the tape attribute data: unexpected length 0x%04x." -#define LTFS11238E "Cannot apply NFC: failed to get output buffer size (%d)." -#define LTFS11115W "Cache manager: failed to fully expand the pool." -#define LTFS12054E "Cannot unformat medium: locate to partition 0, block 0 failed (%d)." -#define LTFS12022I "Unloading medium." -#define LTFS11999E "Cannot load the medium: failed to get capacity data (%d)." -#define LTFS11265E "Cannot parse configuration file: \'default\' directive must be followed by a plug-in type and name." -#define LTFS11255I "Appending a file mark to the data partition." -#define LTFS11045E "Cannot set times: device is not ready." -#define LTFS12064E "Cannot write block: no space left on device." -#define LTFS11222E "Cannot check medium: failed to save index partition append position (%d)." -#define LTFS12031E "Cannot set compression: backend call failed (%d)." -#define LTFS17187E "Unexpected not ready state (%d)." -#define LTFS11090W "Data partition identifier requested on an uninitialized volume." -#define LTFS17084E "XML parser: failed to read extent list from file (%d)." -#define LTFS17258E "A critical error happened while parsing an index (%d). Stop seeking the latest index." -#define LTFS12048E "Cannot read: must open the device first." -#define LTFS11019D "Checking volume consistency." -#define LTFS11150E "Size= rule must contain a valid size." -#define LTFS17242W "Failed to acquire the advisory lock '%s' (%d)." -#define LTFS17054E "Cannot instantiate an index writer direct to tape." -#define LTFS11005I "Mounting the volume." -#define LTFS17086W "Cannot get kernel version." -#define LTFS11295E "Cannot read partition label: check for file mark after the ANSI label failed (%d)." -#define LTFS11316E "Failed to print help message of key manager interface backend." -#define LTFS17041E "XML parser: read failed while looking for a file mark (%d)." -#define LTFS17059E "Index writer: cannot validate extended attribute value (%d)." -#define LTFS11281E "Cannot load messages: failed to get message table (%d)." -#define LTFS17069E "Failed to sync index." -#define LTFS11029E "Cannot mount volume: failed to save the append position for the index partition." -#define LTFS17183E "Error writing XML schema to file '%s' on the disk." -#define LTFS12036E "Seek failed: final position is not what was requested." -#define LTFS11340I "Revalidation process is successfully done. (%s)" -#define LTFS11242E "Cannot convert UTF-8 to UTF-16: failed to get output buffer size (%d)." -#define LTFS11213E "Cannot check medium: seek index failed on the data partition (%d)." -#define LTFS17176E "Cannot read: failed to get data key (%d)." -#define LTFS11148E "More than one non-numeric character follows the size criterion." -#define LTFS17036E "XML parser: expected a text node (received type %d)." -#define LTFS17024E "XML parser: invalid size criterion %s." -#define LTFS17208I "Encountered the last data on the tape (%d). The last read index generation is %d." -#define LTFS11231E "A simple fix is possible to restore the consistency of the tape." -#define LTFS17170E "Failed to parse LTFS dcache configuration rules: invalid option '%s'." -#define LTFS11099E "Cannot format: failed to set the medium compression (%d)." -#define LTFS11225E "Cannot check medium: failed to allocate index data (%d)." -#define LTFS17093E "XML parser: failed to skip tag." -#define LTFS11177E "The ANSI label is too long (%d)." -#define LTFS11030I "Failed to sync volume (%d). Stop the periodic sync thread." -#define LTFS17083E "Traverse(%c): cannot locate to the last index position (%c)." -#define LTFS11147E "Duplicate index criteria option \'%s\'." -#define LTFS17230W "The bar code information is null." -#define LTFS17064D "Sync is done. (%s)." -#define LTFS11325E "Cannot set extended attribute: failed to flush (%d)." -#define LTFS11157E "Cannot specify a name rule without a size rule." -#define LTFS11083E "Cannot write index: failed to generate and write XML data (%d). The medium might be in an inconsistent state; unmount and run ltfsck." -#define LTFS17226W "Cannot update the tape attribute: the maximum (%s) attribute length is %d." -#define LTFS11331E "Failed to load the cartridge (%s)." -#define LTFS17245E "XML writer: failed to flush cached data to the disk (%d)." -#define LTFS11051E "Cannot unlink: failed to format the path (%d)." -#define LTFS11111E "Base64 decoder: empty input." -#define LTFS12026D "Locking medium in the drive." -#define LTFS17199W "Cannot update the tape attribute (type: 0x%04x): %s." -#define LTFS17100E "XML parser: UID on the root directory must be 1." -#define LTFS11183E "Comparing labels: volume UUID mismatch." -#define LTFS17014E "Cannot parse index: failed to create XML parser input buffer." -#define LTFS11128E "Cannot get extended attribute: failed to look for virtual xattrs (%d)." -#define LTFS11193E "Cannot read index: failed to determined medium position (%d)." -#define LTFS17002E "XML parser: duplicate optional tag \'%s\'." -#define LTFS11303E "Data placement rule is too long: '%s'." -#define LTFS17020E "XML parser: invalid schema version \'%s\'." -#define LTFS12018E "Cannot load the medium: failed to lock the medium in the drive (%d)." -#define LTFS17148E "Use ltfsck with the --deep-recovery option." -#define LTFS17098I "Device Name = %s (%d.%d.%d.%d), Vendor ID = %s, Product ID = %s, Serial Number = %s, Product Name =%s." -#define LTFS12021E "Cannot load the medium: failed to get device parameters (%d)." -#define LTFS11160E "Cannot prepare glob cache: failed to prepare name for caseless matching (%d)." -#define LTFS17264I "The index on %s is newer, but MAM shows a permanent write error happened on %s." -#define LTFS17152E "Cannot set WRITE MODE: Failed to load medium (%s, %d)." -#define LTFS11203E "Cannot locate index: failed to space back 2 file marks (%d)." -#define LTFS17126E "Unexpected EOD status (%d, %d)." -#define LTFS12016E "No medium present." -#define LTFS10005E "Null argument (%s) to %s." -#define LTFS17130I "Expected read error occurred." -#define LTFS17146E "EOD of %s(%d) is missing. A deep recovery operation is required." -#define LTFS17190I "Data key is set to enable encryption feature." -#define LTFS17109E "Failed to detect the final index or the final record." -#define LTFS11022I "Restoring volume consistency by writing an index to the index partition." -#define LTFS17263I "Sleep failed while taking the advisory lock '%s' (%d, %d)." -#define LTFS11221E "Medium check failed: detected invalid extents." -#define LTFS12046E "Cannot write file marks: must open the device first." -#define LTFS11204D "No index found. Space back and try again." -#define LTFS17136E "Failed to erase at EOD recovery point." -#define LTFS11272E "Cannot parse configuration file: \'option\' directive must be followed by an option type and LTFS mount option." -#define LTFS11105E "Cannot format: failed to generate LTFS label." -#define LTFS12050E "Cannot format medium: locate to partition 0, block 0 failed (%d)." -#define LTFS11321E "Symbolic link might be replaced by data file. Use ltfsck for recovery." -#define LTFS17096W "The index read from the tape uses a newer version of the LTFS format than the one supported by this software. Some functionality might be unavailable. If this tape is modified, the index downgrades format version to %s from %d.%d.%d." -#define LTFS11012D "Loading the tape." -#define LTFS12060W "Cannot get VCI data: unexpected VCR length 0x%04x." -#define LTFS11008D "Reading partition labels." -#define LTFS17021E "XML parser: unsupported %s version %s." -#define LTFS11261E "Cannot load plug-in: %s." -#define LTFS11049E "Cannot create: path lookup failed (%d)." -#define LTFS11226I "Erasing bad blocks from the index partition." -#define LTFS17206E "Cannot write XML data to file descriptor (%s, %d, %lu)." -#define LTFS11218E "Cannot check medium: failed to write a file mark to the index partition (%d)." -#define LTFS11142E "Cannot list physical xattrs: failed to convert key to system locale (%d)." -#define LTFS17254E "This cartridge cannot be reformatted in the drive (0x%02x, %d)." -#define LTFS11154E "Cannot parse index criteria: error while seeking name rule." -#define LTFS11054E "Cannot rename: failed to format the source path (%d)." -#define LTFS17053E "Cannot generate index data direct to tape: failed to create output buffer." -#define LTFS11291I "Eject successful." -#define LTFS11001E "Cannot instantiate LTFS volume: failed to allocate label data." -#define LTFS17246E "Failed to %s (%d)." -#define LTFS9013E "Cannot specify \"--quiet\" with \"--trace\" or \"--fulltrace\"." -#define LTFS17045E "Label writer: failed to format time." -#define LTFS11039E "Cannot open file: failed to format the path (%d)." -#define LTFS11046E "Cannot set read-only flag: device is not ready." -#define LTFS11088E "Cannot read: failed to get a block from the medium (%d)." -#define LTFS17164E "Cannot reset the capacity proportion: backend call failed (%d)." -#define LTFS17065D "Periodic sync thread initialized." -#define LTFS17113I "Starting EOD recovery: reading an index from (%llu, %llu)." -#define LTFS11194W "Cannot read index: failed to read and parse XML data (%d)." -#define LTFS17032E "XML parser: compression must be \'true\' (1) or \'false\' (0)." -#define LTFS11229E "Cannot restore medium consistency: failed to save index partition append position (%d)." -#define LTFS11310W "Cannot set the traversal mode: failed to get a valid selection %d." -#define LTFS17202E "Failed to wipe the medium (%d)." -#define LTFS17173W "Time conversion is underflowed. (%04d-%02d-%02dT%02d:%02d:%02dZ)." -#define LTFS11173E "Cannot read label (%d): failed to seek to block 0 (partition %lu)." -#define LTFS17207E "Use the ltfsck command with the --salvage-rollback-points option and select the latest index from the list\n Then use the ltfs command with the -o rollback-mount-no-eod option by specifying the generation." -#define LTFS11130E "Cannot list extended attributes: NULL output buffer with a non-zero size." -#define LTFS11319E "Cannot add the key to hash table at %s (%d)." -#define LTFS17234W "The index read from the tape uses a newer version of the LTFS format than the one supported by this software. If this tape is modified, the index downgrades to format version %s from %d.%d.%d." -#define LTFS10000E "Failed to initialize libltfs (%d)." -#define LTFS12013E "Cannot inquire the tape device: backend call failed (%d)." -#define LTFS17145W "Tape backend does not support missing EOD detection." -#define LTFS17217W "%s is out of range in setting time (%s:%llu sec=%lld)." -#define LTFS17135E "Failed to seek to EOD recovery point." -#define LTFS11101E "Cannot format: failed to locate (%d) to partition %u, block 0." -#define LTFS11098E "Cannot format: failed to partition the medium (%d)." -#define LTFS17159I "Process was interruped by user." -#define LTFS17129W "Unexpected read error occurred." -#define LTFS11276W "Ignoring unknown configuration directive \'%s\'." -#define LTFS17010E "Cannot parse XML label from memory." -#define LTFS11248E "Cannot convert system locale to UTF-16: failed to get output buffer size (%d) for '%s'." -#define LTFS11335D "Get physical block position (%d - %d)." -#define LTFS17092E "Index writer: failed to write opaque tags (%s)." -#define LTFS17071I "Unpartitioning the medium." -#define LTFS17003E "XML parser: cannot determine whether tag is empty." -#define LTFS12055E "Cannot unformat medium: backend call failed (%d)." -#define LTFS11271E "Cannot parse configuration file: unknown plug-in type \'%s\' in \'-default\' directive." -#define LTFS11108E "Cannot format: failed to write file mark after XML label (%d) on partition %u." -#define LTFS11116E "Cache manager: failed to grow the pool." -#define LTFS12025D "Releasing device." -#define LTFS11250E "Cannot convert UTF-8 to system locale: failed to get output buffer size (%d)." -#define LTFS11269E "Cannot parse configuration file: line too long." -#define LTFS12032E "Cannot set append position: invalid partition %lu." -#define LTFS11246E "Cannot convert system locale to UTF-16: failed to open converter (%d)." -#define LTFS17051E "Cannot instantiate an index writer to file \'%s\'." -#define LTFS12004E "Device backend does not implement all required methods." -#define LTFS17193I "Stack[%02d]: %p, %s." -#define LTFS10009E "No driver plug-in specified and no default found." -#define LTFS11006E "Cannot read volume: failed to load the tape." -#define LTFS11298E "Cannot read volume: unsupported medium." -#define LTFS11286E "Cannot load messages: failed to open resource bundle (%d)." -#define LTFS11188E "Comparing labels: partition IDs must be distinct." -#define LTFS11216E "Cannot check medium: failed to locate to EOD on the index partition (%d)." -#define LTFS17186E "Called releaseread_mrsw with zero or less count." -#define LTFS17194I "Stack[%02d]: %p." -#define LTFS17044E "Label writer: failed to start the document (%d)." -#define LTFS12042E "Cannot write block: must open the device first." -#define LTFS11026I "Performing a full medium consistency check." -#define LTFS11200E "Cannot locate index: failed to determined medium position (%d)." -#define LTFS17177E "Cannot read: no correct key." -#define LTFS11069E "Cannot write blocks: failed to write an index while switching partitions (%d)." -#define LTFS17031E "XML parser: invalid %s \'%s\'." -#define LTFS9008E "Cannot open output converter (ucnv_open: %d)." -#define LTFS17116I "Detecting the final record in DP." -#define LTFS10013E "Failed to set up signal handler." -#define LTFS11339D "%s volume lock status (%d)." -#define LTFS11059E "Cannot truncate: length must not be negative." -#define LTFS11230I "Writing index(es) to restore consistency." -#define LTFS11016W "The index partition MAM parameter is not usable. Mounting will take longer." -#define LTFS17227I "Tape attribute: %s = %s." -#define LTFS17224W "%s is out of range in creating the index (%lld)." -#define LTFS11174E "Cannot read ANSI label: read failed (%d)." -#define LTFS17237E "WORM related error (%s)." -#define LTFS11146E "Invalid index criteria option \'%s\'." -#define LTFS17079E "Traverse: cannot find target generation %d." -#define LTFS17250I "Information of %lld files are written to the sync list." -#define LTFS11324D "Path changed from '%s' to '%s'." -#define LTFS11050E "Cannot unlink: device is not ready." -#define LTFS11186E "Comparing labels: compression mismatch." -#define LTFS17167E "Cannot read volume: failed to get capacity data (%d)." -#define LTFS11164E "Failed to push file name onto a null stack pointer." -#define LTFS17107E "Version mismatch of MAM, IP=%d, DP=%d." -#define LTFS17017E "XML parser: unexpected top-level tag \'%s\'." -#define LTFS17117E "Failed to search the final index in DP." -#define LTFS11084E "Cannot write index: failed to write a trailing file mark (%d). The medium is in an inconsistent state; unmount and run ltfsck." -#define LTFS11190E "Comparing labels: unknown partition ID." -#define LTFS11306E "Cannot get read-only state of partition: invalid partition identifier." -#define LTFS11179E "Cannot read LTFS label: parsing failed (%d)." -#define LTFS11314E "Cannot format: failed to get data key (%d)." -#define LTFS11158E "Cannot match name: failed to initialize glob cache (%d)." -#define LTFS11126E "Cannot get extended attribute: failed to validate the name (%d)." -#define LTFS17157I "Changing the drive setting %s." -#define LTFS11309E "Cannot parse configuration file: \'-plugin\' directive must be followed by a plug-in type and name." -#define LTFS17229D "Truncate the tape attribute: %s (%s) to %d bytes." -#define LTFS11243E "Cannot convert UTF-8 to UTF-16: failed to fill output buffer (%d)." -#define LTFS12058W "Cannot get VCI data: unexpected ID 0x%04x." -#define LTFS17248W "Failed to open %s (%s)." -#define LTFS17222W "%s is out of range in getting xattr (%s:%llu sec = %lld)." -#define LTFS10004E "Cannot open device \'%s\'." -#define LTFS17125E "Failed to seek to the final record in %s (%d)." -#define LTFS12017E "Cannot load the medium (%d)." -#define LTFS17072E "Cannot unformat: failed to unpartition the medium (%d)." -#define LTFS17182D "Writing schema to disk: UUID='%s', barcode='%s'." -#define LTFS17131I "Recovering EOD at (%llu, %llu) takes about 20 minutes." -#define LTFS17139I "Starting a deep recovery operation: missing EOD." -#define LTFS17027E "XML parser: unsupported extended attribute type \'%s\'." -#define LTFS11033E "Cannot unmount: failed to write an index." -#define LTFS17108E "Cannot find partition id \'%c\'(0x%x)." -#define LTFS17047E "Cannot generate LTFS label: failed to allocate buffer." -#define LTFS17078D "Traverse(%c): cannot find target generation %d (%c)." -#define LTFS11023E "Cannot mount volume: seek to index failed on the index partition." -#define LTFS17185E "Cannot read LTFS label: max transfer length is shorter than max LTFS label length (%d)." -#define LTFS11282E "Cannot load messages: failed to determine first message ID (ures_getByKey: %d)." -#define LTFS11205E "Index partition contains a back pointer, but no index found on the data partition." -#define LTFS12045E "Cannot write block: backend call failed (%d). Dropping to read-only mode." -#define LTFS12010E "Failed to grab the device lock (%s)." -#define LTFS12051E "Cannot format medium: Mode Sense for Medium Partition failed (%d)." -#define LTFS17191I "Data key is cleared to disable encryption feature." -#define LTFS11275E "Cannot parse configuration file: \'plug-in\' directive must be followed by a plug-in type, name, and library path." -#define LTFS17055E "Cannot generate index data direct to tape (%d)." -#define LTFS17022E "XML parser: invalid block size %s." -#define LTFS11223I "Generating lost and found directory." -#define LTFS11093E "Cannot format: failed to load the medium (%d)." -#define LTFS11234E "Cannot validate extended attribute value: code point iteration failed." -#define LTFS11262E "Cannot unload plug-in: %s." -#define LTFS17049E "Cannot instantiate index writer to memory." -#define LTFS17220W "%s is out of range in parsing dentry (%s:%llu %s)." -#define LTFS11332I "Load successful." -#define LTFS11254D "libltfs read from \'%s\': offset = %lld, count = %llu." -#define LTFS11010E "Cannot read volume: failed to set medium compression." -#define LTFS17228I "Tape attribute: %s = 0x%02x." -#define LTFS17085I "Plugin: Loading \"%s\" %s backend." -#define LTFS11151E "Size= rule must contain a digit." -#define LTFS11018D "Done reading MAM parameters." -#define LTFS17007E "Cannot instantiate an LTFS label parser for file \'%s\'." -#define LTFS11002E "Cannot instantiate LTFS volume: failed to allocate index data." -#define LTFS9010E "Invalid option \'%s\'." -#define LTFS11228E "Cannot restore medium consistency: failed to save data partition append position (%d)." -#define LTFS12037E "Cannot seek: backend call failed (%d)." -#define LTFS11343I "Try to write an index on the IP on %s because of a permanent write error on the DP.." -#define LTFS11017W "The data partition MAM parameter is not usable. Mounting will take longer." -#define LTFS11253E "No index found in the medium." -#define LTFS11212E "Cannot create lost and found directory: seek failed (%d)." -#define LTFS17077I "Traverse(%c): find target generation %d (%c)." -#define LTFS11072E "Cannot write blocks: failed to write to the medium (%d)." -#define LTFS11089E "Cannot read: expected %u bytes from the medium, but received %u bytes." -#define LTFS17060E "XML writer: failed to write a block to the medium (%d)." -#define LTFS17110I "The MAM was written by PGA1 or earlier." -#define LTFS11305E "Empty data placement rule in '%s'." -#define LTFS11138E "Cannot remove extended attribute: failed to validate the name (%d)." -#define LTFS11311E "Cannot format: failed to reset capacity proportion (%d)." -#define LTFS17035E "XML parser: expected a text node." -#define LTFS17068I "Syncing index of %s (Reason: %s) %s." -#define LTFS11329E "Failed to recover tape: cannot write the index to an invalid position in the data partition (%lld, %lld, %d)." -#define LTFS17029E "XML parser: invalid UUID %s." -#define LTFS17088W "Cannot get distribution information." -#define LTFS11170E "Failed to read label (%d) from partition 0." -#define LTFS11125E "Cannot get extended attribute: failed to format the name (%d)." -#define LTFS17203W "Truncate the tape attribute: %s (%s) to %d bytes." -#define LTFS11131E "Cannot list extended attributes: failed to format the path (%d)." -#define LTFS17162I "Trying a simple recovery that requires several minutes to complete." -#define LTFS17174E "Key manager interface backend does not implement all required methods." -#define LTFS11104E "Cannot format: failed to write file mark after ANSI label (%d) on partition %u." -#define LTFS17140E "Tape backend does not support missing EOD detection." -#define LTFS11112E "Base64 decoder: invalid character in the input." -#define LTFS12029E "Device is not ready (%d)." -#define LTFS11182E "Comparing labels: tape bar code number mismatch." -#define LTFS17103E "Cannot set PEWS: Mode Select for Device Configuration Extension failed (%d)." -#define LTFS17189I "Device is becoming ready (%d)." -#define LTFS17013E "Cannot parse index: failed to determine medium position (%d)." -#define LTFS11080E "Cannot write index: failed to locate append position on partition %c (%d)." -#define LTFS17265I "Skip writing the index because of %s." -#define LTFS11302E "Data placement rule contains invalid character(s): '%s'." -#define LTFS12019E "Cannot load the medium: failed to determine medium position (%d)." -#define LTFS17118I "Seeking to the final index in %s (%llu, %llu)." -#define LTFS12020E "Cannot load the medium: failed to set device defaults (%d)." -#define LTFS17168E "Cannot read volume: medium is not partitioned." -#define LTFS11163E "Cannot perform matching: failed to open text boundary iterator for filename (%d)." -#define LTFS17046E "Label writer: failed to close the document (%d)." -#define LTFS12033E "Cannot seek to append position: seek failed (%d)." -#define LTFS17239E "Failed to update density (%s) %d." -#define LTFS11257I "No index found in the index partition." -#define LTFS11247E "Cannot convert system locale to UTF-16: failed to set up converter (%d)." -#define LTFS10008E "Failed to load the configuration file (%d)." -#define LTFS17121E "Failed to read the final index in %s (%d)." -#define LTFS11007D "Tape is loaded." -#define LTFS17233E "Failed to kick gcore." -#define LTFS17061E "XML writer: failed to flush cached data to the medium (%d)." -#define LTFS11297E "Cannot read index: failed to allocate index structure (%d)." -#define LTFS12038E "Cannot seek EOD: invalid partition %lu." -#define LTFS11287E "Cannot load messages: failed to register message data (%d)." -#define LTFS17238I "WORM status updated (%s=>%d) '%s'." -#define LTFS11215E "Cannot check medium: failed to locate to EOD on the data partition (%d)." -#define LTFS11027E "Cannot mount volume: medium consistency check failed." -#define LTFS17181E "Cannot reopen device: failed backend reopen call." -#define LTFS11240E "Cannot apply NFD: failed to get output buffer size (%d)." -#define LTFS11201E "Cannot locate index: failed to space back 1 file mark (%d)." -#define LTFS17195E "Cannot get maximum block size : %s (%d)." -#define LTFS17034E "XML parser: invalid time \'%s\' (%d)." -#define LTFS11320E "Cannot search the key from hash table at %s (%d)." -#define LTFS17095W "The index read from the tape uses an old version of the LTFS format. If this tape is modified, the index upgrades format version to %s from %d.%d.%d." -#define LTFS12063W "Cannot set VCI data: backend call failed (%d)." -#define LTFS10014E "Failed to clean up signal handler." -#define LTFS11009E "Cannot read volume: failed to read partition labels." -#define LTFS11237E "Cannot fold string case: failed to fill output buffer (%d)." -#define LTFS11015W "VCR MAM parameter is not usable. Mounting will take longer." -#define LTFS11227I "Preserve existing unreferred data blocks in the data partition, put the latest index at %llu." -#define LTFS17080D "Traverse(%c): find generation %d (%c)." -#define LTFS17026E "XML parser: file size is shorter than extent list." -#define LTFS17081E "Traverse(%c): callback function failed %d (%c)." -#define LTFS11097I "Partitioning the medium." -#define LTFS17257I "0x2f (/) or 0x1f (US) exists in name object (%s): Replace it with '_'." -#define LTFS11141E "Cannot list physical xattrs: failed to generate namespace prefix (%d)." -#define LTFS17247W "Failed to create the path of %s (%s)." -#define LTFS11155E "Cannot parse index criteria: failed to parse size rule (%d)." -#define LTFS17066D "Periodic sync thread uninitialized." -#define LTFS11327E "Failed to seek EOD: seek invalid partition (%d, %d)." -#define LTFS11057E "Cannot rename: path lookup failed for destination (%d)." -#define LTFS11337I "Update index-dirty flag (%d) - %s (0x%p)." -#define LTFS11267E "Cannot get library path: unknown plug-in type \'%s\' or plug-in name \'%s\'." -#define LTFS17050E "Cannot generate index data in memory." -#define LTFS17008E "Cannot parse XML label from file \'%s\'." -#define LTFS11047E "Cannot create: device is not ready." -#define LTFS11290E "Failed to eject the cartridge (%s)." -#define LTFS11117E "Cannot set extended attribute: device is not ready." -#define LTFS17073I "Tape Device list:." -#define LTFS11185E "Comparing labels: block size mismatch." -#define LTFS17016E "Cannot parse index direct from medium." -#define LTFS11167E "Cannot create file or directory: failed to allocate dentry." -#define LTFS17106E "XML parser: UID 0 is reserved." -#define LTFS11191E "Comparing labels: partitions have the same ID \'%c\'." -#define LTFS17232E "Failed to initialize file system component (%d)." -#define LTFS11085E "Cannot read: failed to determine medium position (%d)." -#define LTFS11301E "Unable to read index: cannot duplicate index partition criteria." -#define LTFS17179I "Cleaning cartridge was loaded. (TA=%016llx)." -#define LTFS13003E "Scheduler backend does not implement all required methods." -#define LTFS17114I "Seaching the final index in IP." -#define LTFS11315E "Cannot format: failed to set data key (%d)." -#define LTFS11344I "Tape %s is frozen successfully because of a permanent write error on the DP." -#define LTFS17039E "XML parser: failed to read a block from the medium (%d)." -#define LTFS11118E "Cannot set extended attribute: failed to format the path (%d)." -#define LTFS17154E "Cannot set WRITE MODE: Mode Sense for Device Configuration Extension failed (%d)." -#define LTFS11308E "Commit message must be 65536 bytes or less." -#define LTFS17166D "Coherency Data: %s (%lld, %lld, %lld, %u, %s, %c)." -#define LTFS11121E "Cannot set extended attribute: failed to look up the path (%d)." -#define LTFS12041E "Cannot space file marks: backend call failed (%d)." -#define LTFS11100I "Writing label to partition %c." -#define LTFS10003E "Cannot initialize condition variable (%d)." -#define LTFS12014E "Cannot open device: failed to reserve the device (%d)." -#define LTFS11279E "Cannot write index to partition %c (%d)." -#define LTFS11214E "Cannot check medium: seek index failed on the index partition (%d)." -#define LTFS17165I "Resetting the medium's capacity proportion." -#define LTFS17188I "Device is not ready (%d)." -#define LTFS17144E "The MAM of %s is not usable." -#define LTFS11336I "The attribute does not exist. Ignore the expected error." -#define LTFS11249E "Cannot convert system locale to UTF-16: failed to fill output buffer (%d) for '%s'." -#define LTFS11020E "Cannot mount volume: seek to index failed on the data partition." -#define LTFS11198E "Cannot read index: failed to space forward 1 file mark (%d)." -#define LTFS11270E "Cannot parse configuration file: \'-default\' directive must be followed by a plug-in type." -#define LTFS17192W "Cannot read: medium is encrypted." -#define LTFS12024E "Cannot reserve device: backend call failed (%d)." -#define LTFS17023E "XML parser: invalid generation number %s." -#define LTFS11053E "Cannot rename: device is not ready." -#define LTFS11263E "Cannot resolve plug-in operation interface: %s." -#define LTFS10019E "Error on runcommand (%s, %d, %d)." -#define LTFS11123E "Cannot get extended attribute: NULL value with non-zero size." -#define LTFS11333I "A cartridge with write-perm error is detected on %s. Seek the newest index (IP Gen = %llu, VCR = %llu) (DP Gen = %llu, VCR = %llu) (VCR = %llu)." -#define LTFS10021D "xattr: %s: %s (%d, %d)." -#define LTFS11220E "Medium check failed: extra blocks detected. Run ltfsck." -#define LTFS17240I "Density code is updated to x%02x." -#define LTFS11152E "Cannot parse index criteria: rules are invalid." -#define LTFS11003E "Cannot retrieve device capacity data (%d)." -#define LTFS11268E "Cannot open configuration file \'%s\' (%d)." -#define LTFS9011E "Logging initialization failed." -#define LTFS11040E "Cannot open file: path lookup failed (%d)." -#define LTFS11293E "Cannot load messages for libltfs (%d)." -#define LTFS11252D "libltfs write to \'%s\': offset = %lld, count = %llu." -#define LTFS11342E "Failed to get the volume lock status (%d)." -#define LTFS12034E "Cannot get maximum device blocksize: backend call failed (%d)." -#define LTFS11283E "Cannot load messages: failed to determine first message ID (ures_getInt: %d)." -#define LTFS11334I "Remove extent : %s (%llu, %llu)." -#define LTFS11135E "Cannot remove extended attribute: device is not ready." -#define LTFS11312I "Revalidation process of the medium is starting. (%s)" -#define LTFS17178E "Cannot read: failed to set data key (%d)." -#define LTFS17030E "XML parser: failed to normalize %s \'%s\'." -#define LTFS17204W "Cannot set: unkown tape attribute type(0x%04x): %s." -#define LTFS17260I "Sleep was interrupted by a signal while taking the advisory lock '%s'." -#define LTFS11233I "Updating MAM coherency data." -#define LTFS17091E "Cannot save tag: libxml2 could not return text for this node." -#define LTFS11169E "Cannot read labels: failed to allocate label data (%d)." -#define LTFS17141I "Both EODs are detected. A deep recovery operation is unnecessary." -#define LTFS17236I "Wrote index of %s (%c, %s)." -#define LTFS9014E "Cannot create work directory \'%s\': %s." -#define LTFS11175E "Cannot read ANSI label: expected 80 bytes, but received %d." -#define LTFS17172W "Time conversion is overflowed. (%04d-%02d-%02dT%02d:%02d:%02dZ)." -#define LTFS17171E "Failed to parse LTFS dcache configuration rules: invalid value '%d' for option '%s'." -#define LTFS11145E "Cannot get attribute %s: failed to generate the time string." -#define LTFS11132E "Cannot list extended attributes: failed to look up the path (%d)." -#define LTFS17223W "%s is out of range in creating the label (%lld)." -#define LTFS11159E "Cannot match name: failed to prepare for caseless matching (%d)." -#define LTFS17253E "Cannot get tape parameters: %s (%d)." -#define LTFS11323D "Symlink EA prefix valuse is (%s)." -#define LTFS9006E "Cannot load resource \"fallback_messages\" (ures_getByKey: %d)." -#define LTFS17143I "EOD of %s(%d) is missing." -#define LTFS12028D "Unlocking medium." -#define LTFS9016E "Cannot set the LANG environment variable." -#define LTFS11113E "Base64 decoder: input length is not a multiple of 4." -#define LTFS11181E "Cannot read partition label: failed to find the trailing file mark." -#define LTFS17012E "Cannot parse index from file \'%s\'." -#define LTFS17102E "Cannot set PEWS: Mode Sense for Device Configuration Extension failed (%d)." -#define LTFS17266I "Skip setting the append only mode because the drive doesn't seem to support it." -#define LTFS11081E "Cannot write index: failed to determine medium position (%d)." -#define LTFS11998W "Unable to delete dentry '%s': it still has outstanding references." -#define LTFS17028E "XML parser: base64 decoding failed." -#define LTFS12023D "Reserving device." -#define LTFS11114E "Cache manager: failed to initialize the pool." -#define LTFS11211E "Cannot populate lost and found directory: failed to allocate file data." -#define LTFS17048E "Cannot generate index data in memory: failed to allocate buffer." -#define LTFS12059W "Cannot get VCI data: unexpected length 0x%04x." -#define LTFS11244E "Cannot convert UTF-16 to UTF-8: failed to get output buffer size (%d)." -#define LTFS10007E "Not yet implemented (%s)." -#define LTFS17249I "Information of %lld files are written to the offset cache." -#define LTFS17221W "%s is out of range in setting xattr (%s:%llu %s)." -#define LTFS17132E "Failed to get current position." -#define LTFS17124I "Seeking to the final record in %s (%llu, %llu)." -#define LTFS11032D "Unmounting the volume." -#define LTFS17196W "Cannot retrieve the tape attribute data: unexpected ID 0x%04x." -#define LTFS17184E "Error changing index cache file's permission (%d)." -#define LTFS11259I "Cannot move directory: Directory move is prohibited because of a MacFUSE bug." -#define LTFS11284E "Cannot resolve plug-in message bundle interface: %s." -#define LTFS11024E "Cannot mount volume: read index failed on the index partition." -#define LTFS17056E "XML writer: cannot format time (gmtime failed)." -#define LTFS11206E "Index partition back pointer is invalid." -#define LTFS17138I "Recovered EOD status successfully (%d)." -#define LTFS12044E "Cannot write a %u-byte block: maximum block size is %lu." -#define LTFS11014D "Reading MAM parameters." -#define LTFS17000E "XML parser: not all required tags found in \'%s\'." -#define LTFS12062W "Cannot get VCI data: unexpected Application Specific Info signature." -#define LTFS11236E "Cannot fold string case: failed to get output buffer size (%d)." -#define LTFS11096E "Cannot format: requested block size is %lu bytes, but the device only supports %u." -#define LTFS11224E "Cannot restore medium consistency: failed to generate lost and found (%d)." -#define LTFS17094E "XML parser: comment field is longer than 64 KiB." -#define LTFS17231E "Cannot %s the tape attribute: %s." -#define LTFS17082E "Traverse(%c): cannot locate to the first index position (%c)." -#define LTFS11176E "The ANSI label indicates this is not an LTFS volume." -#define LTFS11140E "Cannot remove extended attribute: failed to look up the xattr (%d)." -#define LTFS9015W "Setting the locale to 'en_US.UTF-8'. If this setting is wrong, set the LANG environment variable before starting %s." -#define LTFS11326E "Cannot write index: failed to flush buffered data (%d)." -#define LTFS11156E "Cannot parse index criteria: error while seeking size rule." -#define LTFS11330I "Loading cartridge." -#define LTFS17076E "Traverse(%c): cannot locate to the next index position (%c)." -#define LTFS9001E "Failed to parse command line options." -#define LTFS17244E "XML writer: failed to write a block to the disk (%d)." -#define LTFS11056E "Cannot rename: path lookup failed for source (%d)." -#define LTFS11264E "Cannot get plug-in operations." -#define LTFS17256I "An unexpected character exists in the decoded name object (%s). Revert to the encoded character." -#define LTFS11239E "Cannot apply NFC: failed to fill output buffer (%d)." -#define LTFS17019E "XML parser: no schema version found." -#define LTFS11162E "Cannot perform matching: failed to open text boundary iterator for criteria (%d)." -#define LTFS17105E "Cannot load the medium: failed to get programmable early warning size (%d)." -#define LTFS17111I "The MAM was written by PGA2 or later." -#define LTFS11304E "Failed to validate data placement rule (%d)." -#define LTFS11178E "Cannot read LTFS label: read failed (%d)." -#define LTFS11086E "Cannot read: failed to locate (%d) to partition %c, record %llu." -#define LTFS17225W "%s is out of range in creating dentry (%lld)." -#define LTFS11139E "Cannot remove extended attribute: failed to look up the path (%d)." -#define LTFS11196W "Cannot read index: self-pointer mismatch." -#define LTFS17101E "XML parser: UID 1 is reserved for the root directory." -#define LTFS11328E "Failed to seek index: seek invalid partition (%c, %c)." -#define LTFS17200E "XML parser: cannot save tag, libxml2 workaround failed (%s)." -#define LTFS11124E "Cannot get extended attribute: failed to format the path (%d)." -#define LTFS11171E "Failed to read label (%d) from partition 1." -#define LTFS17089I "Distribution: %s." -#define LTFS17161I "EOD information (%s) is corrupted." -#define LTFS17175E "Cannot read: failed to get data key identifier (%d)." -#define LTFS11149E "Invalid size criterion multiplier \'%c\'." -#define LTFS17151E "Cannot set WRITE MODE: Failed to unload medium (%d)." -#define LTFS11136E "Cannot remove extended attribute: failed to format the path (%d)." -#define LTFS11202E "Cannot locate index: failed to space forward 1 file mark (%d)." -#define LTFS11278I "Writing index to partition %c." -#define LTFS12040E "Cannot parse backend options: backend call failed (%d)." -#define LTFS17127I "Detecting the final record to read by hitting read error takes about 20 minutes." -#define LTFS10002E "Cannot initialize mutex (%d)." -#define LTFS11082E "Cannot write index: failed to write file mark (%d)." -#define LTFS11289I "Ejecting cartridge." -#define LTFS17137E "Failed to recover EOD status (%d)." -#define LTFS17147I "Attempting to mount a cartridge without EOD status check." -#define LTFS17006W "XML parser: ignoring unrecognized tag \'%s\' inside \'%s\'." -#define LTFS17198D "Cannot get the tape attribute: 0x%04x (%s)." -#define LTFS17037E "XML parser: failed to read from XML stream." -#define LTFS11021E "Cannot mount volume: read index failed on the data partition." -#define LTFS11199E "Cannot locate index: failed to locate to EOD (%d)." -#define LTFS12057W "Cannot get VCI data: read attribute failed (%d)." -#define LTFS12008E "Cannot allocate device data: failed to initialize mutex (%d)." -#define LTFS12047E "Cannot write file marks: backend call failed (%d). Dropping to read-only mode." -#define LTFS11273E "Cannot parse configuration file: \'%s\' directive must be followed by a valid %s name." -#define LTFS11106E "Cannot format: failed to write XML label (%d) on partition %u." -#define LTFS17218W "%s is out of range in parsing the label (%s)." -#define LTFS11346E "Tape %s is frozen by the previous index on the tape. Some metadata will be lost during the unmount operation." -#define LTFS11256I "Appending a file mark to the index partition." -#define LTFS12030E "Cannot get capacity data: backend call failed (%d)." -#define LTFS11091W "Index partition identifier requested on an uninitialized volume." -#define LTFS17120I "Reading the final index in %s (%llu, %llu)." -#define LTFS17259I "Recover an index on %s from (%c, %llu)." -#define LTFS11209E "Cannot create lost and found directory: failed to allocate directory data." -#define LTFS17062E "XML writer: tried to write a directory as a file." -#define LTFS12049E "Cannot read: backend call failed (%d)." -#define LTFS11004E "Cannot take the device lock (%s)." -#define LTFS11296E "Cannot read partition label: failed to find a file mark after the ANSI label." -#define LTFS17009E "Cannot instantiate an LTFS label parser for a memory buffer." -#define LTFS17058E "Index writer: failed to close the document (%d)." -#define LTFS17040E "XML parser: failed to space back 1 file mark." -#define LTFS17180E "File %s has both of symbolic link and extents." -#define LTFS12039E "Cannot seek EOD: backend locate call failed (%d)." -#define LTFS11280E "Unknown default %s \'%s\'." -#define LTFS11241E "Cannot apply NFD: failed to fill output buffer (%d)." -#define LTFS11028D "Consistency check finished." -#define LTFS17070I "Synced index of %s (%d) %s." -#define LTFS11313E "Medium revalidation failed (%d). Unmount the tape before continuing. (%s)" -#define LTFS11062E "Cannot truncate: device is not ready." -#define LTFS11067E "Cannot write blocks: invalid partition identifier." -#define LTFS17112I "Starting EOD recovery (GA/PGA1)." -#define LTFS17033E "XML parser: invalid partition \'%s\'." -#define LTFS11232E "Rerun the consistency check with simple fixes enabled." -#define LTFS11168E "Cannot allocate index data: failed to allocate root dentry." -#define LTFS17205E "Cannot set the tape attribute (type: 0x%04x): %s ." -#define LTFS17235I "Writing index of %s to %c (Reason: %s, %lld files) %s." -#define LTFS11318W "Unknown log level (%d), forced the level to (%d)." -#define LTFS17252W "Invalid value (%s) in the %s tag in UID %lld." -#define LTFS17063W "Periodic sync thread failed to flush file data to the medium. Data might be lost (%s)." -#define LTFS11172E "Cannot verify labels: comparison failed (%d)." -#define LTFS11133E "Cannot list extended attributes: failed to list real xattrs (%d)." -#define LTFS17243W "Failed to release the advisory lock (%d)." -#define LTFS17090W "sysctl system call failed (%s)." -#define LTFS11013D "Partition labels are valid." -#define LTFS11322D "Makeing a symlink '%s' target '%s'." -#define LTFS17042E "XML writer: error creating tag (%s)." -#define LTFS12027E "Cannot lock medium in the drive: backend call failed (%d)." -#define LTFS11110E "Cannot get current time (%d)." -#define LTFS11011E "Cannot read volume: block size is %lu, but the device only supports a block size of %u." -#define LTFS11184E "Comparing labels: format time mismatch." -#define LTFS17015E "Cannot parse index: failed to create XML reader." -#define LTFS11166E "Cannot allocate index data: failed to initialize mutex (%d)." -#define LTFS11288W "No end ID found for this message bundle: assigning 1000 message IDs." -#define LTFS11192E "Comparing labels: invalid bar code number." -#define LTFS17001E "XML parser: duplicate required tag \'%s\'." -#define LTFS17115E "Failed to search the final index in IP." -#define LTFS11129E "Cannot get extended attribute: failed to look up the xattr (%d)." -#define LTFS17043E "Cannot instantiate an LTFS label writer to memory." -#define LTFS11034I "Volume unmounted successfully." -#define LTFS13004E "Dentry cache backend does not implement all required methods." -#define LTFS17261I "The kernel detected a false deadlock retry to acquire the advisory lock '%s' (%d)." -#define LTFS17038E "XML parser: unexpected end of XML stream." -#define LTFS11119E "Cannot set extended attribute: failed to format the name (%d)." -#define LTFS17075E "Traverse(%c): cannot read index from %d (%c)." -#define LTFS17149E "Cannot erase: backend call failed (%d)." -#define LTFS17099E "Failed to spawn the periodic sync thread (%d)." -#define LTFS17155E "Cannot set WRITE MODE: Mode Select for Device Configuration Extension failed (%d)." -#define LTFS11161E "Cannot match name: match function failed (%d)." -#define LTFS11120E "Cannot set extended attribute: failed to validate the name (%d)." -#define LTFS13019I "FCFS I/O scheduler initialized." -#define LTFS13020I "FCFS I/O scheduler uninitialized." -#define LTFS30267W "WWPID of reservation: x%02x%02x%02x%02x%02x%02x%02x%02x (%s)." -#define LTFS30277W "Detect ENOMEM in ioctl() call. Wait 3 seconds and retry (%d)." -#define LTFS30222W "Received low space warning (early warning) in %s." -#define LTFS30218D "Read block: underrun in illegal length. residual = %d, actual = %d." -#define LTFS30283W "Skip back for %s operation is failed (%d). (%u, %llu), (%u, %llu)." -#define LTFS30247I "No alternate device is found for drive %s." -#define LTFS30210I "Cannot open device: failed to open %s (%d)." -#define LTFS30234I "Cannot get log page 0x%02x (%d) in %s." -#define LTFS30207I "Vendor ID is %s." -#define LTFS30257D "Transferring dump data." -#define LTFS30223W "Received low space warning (programmable early warning) in %s." -#define LTFS30276I "Raw capacity info of partition %d (%llu - %llu) %s." -#define LTFS30280W "Unexpected current position for %s operation (%d). (%u, %llu), (%u, %llu) ." -#define LTFS30204D "%s (0x%02x) expected error %d." -#define LTFS30244I "CDB unexpected status: H = 0x%02x, D = 0x02%x" -#define LTFS30238W "Cannot get EOD status: failed to parse the log page." -#define LTFS30213I "Unsupported Drive \'%s\' / \'%s\'." -#define LTFS30233I "Cannot read attribute (%d)." -#define LTFS30293I "Changer %s was reserved from another node (%s)." -#define LTFS30392D "Backend %s %s." -#define LTFS30268I "Retry to reserve from key registration (%s)." -#define LTFS30260W "Cannot retrieve drive dump: wrote %d bytes out, expected %d." -#define LTFS30256D "Total number of transfers is %d." -#define LTFS30228I "Unsupported cartridge (0x%x, 0x%x)." -#define LTFS30265W "Failed to get medium type code: medium type check is skipped." -#define LTFS30216W "Length mismatch is detected. (Act = %d, resid = %d, resid_sense = %d)." -#define LTFS30291I "Changer %s was reserved from this node and can be reserved from the current path." -#define LTFS30271I "Successfully reopened drive %s with the same path." -#define LTFS30201D "CDB check condition: sense = %06x, %s." -#define LTFS30249I "Opening another path for drive %s on %s." -#define LTFS30285I "The reserved buffer size of %s is %d." -#define LTFS30236I "Unexpected parameter size for getting active CQ loss write (%d)." -#define LTFS30209I "Opening a device through sg-ibmtape driver (%s)." -#define LTFS30220I "Drive cleaning required." -#define LTFS30241E "Invalid scsi_lbprotect option: %s." -#define LTFS30279I "Retrying the %s operation at (%u, %llu) with skip back, because the command was already run (%u, %llu)." -#define LTFS30397D "Backend %s: (%llu, %llu) %s." -#define LTFS30255D "Total dump data length is %lld." -#define LTFS30282W "Unexpected position after skip back for a %s operation. (%u, %llu), (%u, %llu)." -#define LTFS30399I "sg tape backend for IBM tape options:\n -o devname= tape device (default=%s)\n" -#define LTFS30266W "The drive is already reserved: %s (%s)." -#define LTFS30229I "Cannot get remaining capacity: get log page 0x%02x failed (%d)." -#define LTFS30290I "Changer %s isn't reserved from any nodes." -#define LTFS30270I "A power-on-reset happened on drive %s." -#define LTFS30219D "Read block: file mark detected." -#define LTFS30221E "Logical block protection Error on read." -#define LTFS30211I "Cannot %s blocking mode (%d)." -#define LTFS30235I "Cannot parse the log page 0x%02x in %s." -#define LTFS30206I "Cannot open device: inquiry failed (%d)." -#define LTFS30254W "Cannot retrieve drive dump: failed to create dump file (%d)." -#define LTFS30286I "Failed to get the cartridge status. The cartridge is not loaded." -#define LTFS30278I "Retrying the %s operation at (%u, %llu)." -#define LTFS30246I "Connection down is detected, try to reconnect (%s)." -#define LTFS30214I "Firmware revision is %s." -#define LTFS30263I "%s returns %s (%d) %s." -#define LTFS30287I "Received an unknown sense code %06x." -#define LTFS30230I "Cannot parse remaining capacity (0x%x, %d)." -#define LTFS30203I "CDB unexpected status: S = 0x%02x, M = 0x02%x" -#define LTFS30253I "Saving drive dump to %s." -#define LTFS30226I "A long data wipe is in progress. (%d minutes passed)." -#define LTFS30273I "Cannot %s device flag (%d)." -#define LTFS30243E "Encryption feature is not supported on the drive: %d." -#define LTFS30248I "Drive serial is not matched. Actual: %s, Expected %s." -#define LTFS30217I "Read block: overrun in illegal length. residual = %d, actual = %d." -#define LTFS30284I "Cannot get the reserved buffer size of %s." -#define LTFS30264E "%s returns msg = NULL (%d) %s." -#define LTFS30200I "Failed to execute SG_IO ioctl, opcode = %02x (%d)." -#define LTFS30208I "Product ID is %s." -#define LTFS30252I "Logical block protection is disabled." -#define LTFS30227I "A long data wipe is in progress. %d %%." -#define LTFS30272I "Drive %s is successfully reserved." -#define LTFS30398D "Backend %s: (%llu, %llu) FM = %llu %s." -#define LTFS30240I "Cannot open directory (/dev)." -#define LTFS30237W "Cannot get EOD status: failed to get log page 0x%02x (%d)." -#define LTFS30396D "Backend %s: %llu %s." -#define LTFS30394D "Backend %s: %zu %s." -#define LTFS30259W "Cannot retrieve drive dump: failed to write to dump file (%d)." -#define LTFS30232I "Cannot get remaining capacity: failed to parse the log page." -#define LTFS30205I "%s (0x%02x) returns %d." -#define LTFS30281W "Position confirmation for %s operation retry is failed (%d). (%u, %llu), (%u, %llu)." -#define LTFS30275I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." -#define LTFS30245D "Tape device returns %d, ignore for buffered sense cleaning." -#define LTFS30212I "Cannot get drive identifier of %s." -#define LTFS30261I "Taking drive dump in buffer." -#define LTFS30224D "EOD detected (%s): ignore sense." -#define LTFS30393D "Backend %s: %d %s." -#define LTFS30292I "Changer %s was reserved from this node but failed to reserve from the current path." -#define LTFS30269I "Successfully reopened drive %s with another path. Preempting the reservation." -#define LTFS30288I "Opening a tape device for drive serial %s." -#define LTFS30251I "Logical block protection is enabled." -#define LTFS30395D "Backend %s: %zu bytes %s." -#define LTFS30258W "Cannot retrieve drive dump: failed to read buffer (%d)." -#define LTFS30262I "Forcing drive dump." -#define LTFS30239W "Cannot get EOD status: value length or partition number is wrong %d - (%d, %d)." -#define LTFS30231I "Unexpected field in remaining capacity (0x%x)." -#define LTFS30202D "CDB %s." -#define LTFS30274I "Pseudo-error on %s." -#define LTFS30250I "Opened the SCSI tape device %d.%d.%d.%d (%s)." -#define LTFS30225I "Unrecognized space type." -#define LTFS30215I "Drive serial is %s." -#define LTFS30242E "Encryption method of the drive is not AME but %s (0x%02X)." -#define LTFS30289I "Cannot fetch the reservation key (%s, %d)." -#define LTFS14094E "Cannot get mount point (%d)." -#define LTFS14041D "FUSE mkdir \'%s\'." -#define LTFS14114E "Cannot initialize the open file table." -#define LTFS14046D "FUSE rename \'%s\' to \'%s\'." -#define LTFS14031D "FUSE getattr for \'%s\'." -#define LTFS14009E "The backend \'%s\' does not have a default device. Specify one using the \"-o devname=\" option." -#define LTFS14066E "Sync time should be a positive value." -#define LTFS14049D "FUSE read \'%s\' (offset=%lld, count=%zu)." -#define LTFS14039D "FUSE chmod \'%s\'." -#define LTFS14001E "Cannot enable FUSE option %s (%d)." -#define LTFS14422I " -o rules= Rules for choosing files to write to the index partition.\n" -#define LTFS14079E "Invalid uid \'%s\' (must be a positive integer or valid user name)." -#define LTFS14015W "Volume does not allow index criteria override. Ignoring user-specified criteria." -#define LTFS14410I " -o umask=M Set file permissions (octal)" -#define LTFS14026D "Cannot read directory: no buffer space." -#define LTFS14463I " -o scsi_append_only_mode= Set the tape device append-only mode (default=on)" -#define LTFS14115E "Invalid scsi_append_only_mode option: %s." -#define LTFS14057E "Failed to load kmi plug-in (%d)." -#define LTFS14443I " -o force_mount_no_eod Skip EOD existence check when mounting (read-only mount)\n" -#define LTFS14030D "FUSE fgetattr for \'%s\'." -#define LTFS14054E "Failed to load tape drive plug-in (%d)." -#define LTFS14077I "The cartridge will be mounted as read-only." -#define LTFS14067E "Failed to convert the sync time because it is too big or too small (%s)." -#define LTFS14413I " -o config_file= Configuration file (default: %s)" -#define LTFS14436I " -o device_list Show available tape devices" -#define LTFS14006E "Invalid umask \'%s\' (must be 3 octal digits, such as 022)." -#define LTFS14109E "Minimum pool size must be a positive number." -#define LTFS14038D "FUSE set times \'%s\'." -#define LTFS14423I " -o quiet Disable informational messages (same as verbose=1)" -#define LTFS14403I " -o devname= Tape device" -#define LTFS14078I "Medium is Read-Only in this device." -#define LTFS14112I "Invoke 'mount' command to check the result of final setup." -#define LTFS14072I "Rollback mount is specified. Mounting as read-only at generation %d." -#define LTFS14419I " -o dmask= Override directory permission mask (3 octal digits, default: 000)" -#define LTFS14043D "FUSE ftruncate \'%s\' (length=%lld)." -#define LTFS14461I " -o symlink_type= Specify symbolic link type (default: posix)\n" -#define LTFS14405I " -o trace Enable diagnostic output (same as verbose=3)" -#define LTFS14104I "Launched by \"%s\"." -#define LTFS14064I "Sync type is \"%s\"." -#define LTFS14052D "FUSE listxattr \'%s\'." -#define LTFS14033D "FUSE open directory \'%s\'." -#define LTFS14013E "Cannot mount the volume." -#define LTFS14074I "Data partition has no space to write index. Mounting as read-only." -#define LTFS14420I " -o min_pool_size= Minimum write cache pool size. Cache objects are 1 MB each (default: %d)" -#define LTFS14003E "min_pool_size (%d) cannot be greater than max_pool_size (%d)." -#define LTFS14416I " -o iosched_backend= I/O scheduler implementation to use (default: %s, use \"none\" to disable)" -#define LTFS14093E "Unknown type of symbolic link (%s)." -#define LTFS14024E "Cannot get extended attribute: position option must be zero." -#define LTFS14040D "FUSE create file \'%s\'." -#define LTFS14113I "Specified mount point is listed if succeeded." -#define LTFS14071I "Medium has no space to write data. Mounting as read-only." -#define LTFS14418I " -o fmask= Override file permission mask (3 octal digits, default: 000)" -#define LTFS14095I "Set the tape device write-anywhere mode to avoid cartridge ejection." -#define LTFS14008E "Invalid dmask \'%s\' (must be 3 octal digits, such as 022)." -#define LTFS14402I " -o devname= Tape device (default: %s)" -#define LTFS14061E "Unknown type of sync (%s)." -#define LTFS14032D "FUSE open file \'%s\' (%s)." -#define LTFS14055E "Failed to load I/O scheduler plug-in (%d)." -#define LTFS14089E "Could not initialize the kmi plug-in." -#define LTFS14065E "Unexpected sync type (%d)." -#define LTFS14012E "Tape backend option parsing failed." -#define LTFS14000I "LTFS starting, %s version %s, log level %d." -#define LTFS14421I " -o max_pool_size= Maximum write cache pool size. Cache objects are 1 MB each (default: %d)" -#define LTFS14456I " -o capture_index Capture latest index to work directory at unmount" -#define LTFS14411I " -o uid=N Set file owner" -#define LTFS14080E "Invalid gid \'%s\' (must be a positive integer or valid group name)." -#define LTFS14092I "Symbolic link type is (%s)." -#define LTFS14035D "FUSE release file \'%s\'." -#define LTFS14062I "Sync time is 0. Set sync type \"none\"." -#define LTFS14407I " -V, --version Output version information and exit" -#define LTFS14019I "Medium is write protected. Mounting read-only." -#define LTFS14106I "GCC version is %s." -#define LTFS14058I "%s version %s." -#define LTFS14045D "FUSE rmdir \'%s\'." -#define LTFS14110E "Maximum pool size must be a positive number." -#define LTFS14048D "FUSE write \'%s\' (offset=%lld, count=%zu)." -#define LTFS14050D "FUSE setxattr \'%s\' (name=\'%s\', size=%zu)." -#define LTFS14005E "Path \'%s\' exists, but is not a directory." -#define LTFS14011E "Cannot allocate LTFS volume structure." -#define LTFS14029I "Ready to receive file system requests." -#define LTFS14414I " -o atime Update index if only access times have changed" -#define LTFS14441I " -o verbose= Override output verbosity directly (default: %d)" -#define LTFS14091E "Invalid generation for rollback mount is specified. %s." -#define LTFS14034D "FUSE release directory \'%s\'." -#define LTFS14056E "No driver plug-in configured and no default found." -#define LTFS14018E "Cannot get read-only status of the medium." -#define LTFS14063I "Sync type is \"%s\", Sync time is %ld sec." -#define LTFS14404I " -o work_directory= LTFS work directory (default: %s)" -#define LTFS14042D "FUSE truncate \'%s\' (length=%lld)." -#define LTFS14022E "Cannot flush file data to the medium. Data might be lost (%s)." -#define LTFS14053D "FUSE removexattr \'%s\' (name=\'%s\')." -#define LTFS14023E "Cannot set extended attribute: position option must be zero." -#define LTFS14111I "Initial setup completed successfully." -#define LTFS14073I "Index partition has no space to write index. Mounting as read-only." -#define LTFS14427I " -o sync_type= Specify sync type (default: time@5)\n should be specified as follows:\n time@min: LTFS attempts to write an index each 'min' minutes.\n min should be a decimal number from 1 to %ld\n" -#define LTFS14417I " -o umask= Override default permission mask (3 octal digits, default: 000)" -#define LTFS14002E "Cannot set up permissions (%d)." -#define LTFS14090E "KMI backend option parsing failed." -#define LTFS14467I " -o syslogtrace Enable diagnostic output to stderr and syslog(same as verbose=303)" -#define LTFS14437I " -o rollback_mount= Attempt to mount on previous index generation (read-only mount)" -#define LTFS14116E "This medium is not supported (%d)." -#define LTFS14027E "Cannot read directory: convert to system locale failed (%s, %d)." -#define LTFS14037D "FUSE flush \'%s\'." -#define LTFS14047D "FUSE readdir \'%s\'." -#define LTFS14060E "Failed to convert the sync time (%s)." -#define LTFS14076I "Attempting to mount the cartridge without EOD existence check." -#define LTFS14448I " -o release_device Clear device reservation (should be specified with -o devname" -#define LTFS14007E "Invalid fmask \'%s\' (must be 3 octal digits, such as 022)." -#define LTFS14412I " -o gid=N Set file group" -#define LTFS14017E "Cannot parse data placement rules (%d)." -#define LTFS14408I " -h, --help Display this help and exit" -#define LTFS14400I "usage: %s mountpoint [options]" -#define LTFS14424I " -o fulltrace Enable full call tracing (same as verbose=4)" -#define LTFS14406I " -a Advanced help, including standard FUSE options" -#define LTFS14036D "FUSE fsync \'%s\'." -#define LTFS14105I "%s." -#define LTFS14455I " -o kmi_backend= Key manager interface implementation to use (default: %s, use \"none\" to disable)" -#define LTFS14044D "FUSE unlink \'%s\'." -#define LTFS14075E "Cannot set up tape drive." -#define LTFS14004E "Cannot create work directory (%d)." -#define LTFS14051D "FUSE getxattr \'%s\' (name=\'%s\')." -#define LTFS14415I " -o tape_backend= tape backend to use (default: %s)" -#define LTFS14016E "Cannot format data placement rules (%d)." -#define LTFS14409I "FUSE options:" -#define LTFS14440I " -o noatime Do not update index if only access times have changed (default)" -#define LTFS14439I " -o noeject Do not eject the cartridge after unmount (default)" -#define LTFS14401I "LTFS options:" -#define LTFS14068E "Specified sync time is too big or too small (%ld)." -#define LTFS14028W "Cannot initialize I/O scheduler. The scheduler is disabled. Performance might decrease and memory usage might increase." -#define LTFS14425I " -o eject Eject the cartridge after unmount" -#define LTFS31395D "Backend %s: %zu bytes %s" -#define LTFS31287I "Taking drive dump in buffer" -#define LTFS31282D "Transferring dump data" -#define LTFS31284W "Cannot retrieve drive dump: failed to write to dump file (%d)" -#define LTFS31280D "Total dump data length is %lld." -#define LTFS31252I "A long data wipe is in progress. %d %%" -#define LTFS31278I "Saving drive dump to %s" -#define LTFS31253W "Failed to get medium type code: medium type check is skipped" -#define LTFS31246W "Received low space warning (programmable early warning) in %s" -#define LTFS31285D "Transfer %d: wrote %d bytes" -#define LTFS31268I "Cannot allocate memory in %s" -#define LTFS31248D "EOD detected (%s): ignore sense" -#define LTFS31262I "Cannot parse the log page 0x%02x in %s" -#define LTFS31219I "Option parsing for the CAM backend failed (%d)" -#define LTFS31272I "Logical block protection is disabled" -#define LTFS31260I "Cannot read attribute (%d)" -#define LTFS31235I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." -#define LTFS31399I "CAM backend options:\n" -#define LTFS31209D "IOCTL: no sense info" -#define LTFS31225I "Cannot open device \'%s\' (%d)" -#define LTFS31286W "Cannot retrieve drive dump: wrote %d bytes out, expected %d" -#define LTFS31237I "Read block: overlength condition is detected. residual = %d, actual = %d" -#define LTFS31270E "Encryption feature is not supported on the drive: %d" -#define LTFS31259I "Cannot get remaining capacity: loop index error (%d)" -#define LTFS31255I "Unsupported cartridge (0x%x, 0x%x)" -#define LTFS31264W "Cannot get EOD status: failed to get log page 0x%02x (%d)" -#define LTFS31223I "Opening a device through CAM driver (%s)" -#define LTFS31257I "Cannot get remaining capacity: get log page 0x%02x failed (%d)" -#define LTFS31397D "Backend %s: (%llu, %llu) %s" -#define LTFS31393D "Backend %s: %d %s" -#define LTFS31207D "IOCTL: pos = 0x%02x%02x%02x%02x %s" -#define LTFS31211D "IOCTL: %s %d expected error %d. retry the operation." -#define LTFS31229I "Vendor ID is %s" -#define LTFS31239E "Logical block protection Error on read" -#define LTFS31251I "A long data wipe is in progress. (%d minutes passed)" -#define LTFS31213I "Error on %s: %s (%d) %s" -#define LTFS31269E "Encryption method of the drive is not AME but %s (0x%02X)" -#define LTFS31265W "Cannot get EOD status: failed to parse the log page" -#define LTFS31226W "Device \'%s\' must be opened in read-only mode" -#define LTFS31281D "Total number of transfers is %d." -#define LTFS31267I "Unexpected parameter size for getting active CQ loss write (%d)" -#define LTFS31232I "Firmware revision is %s" -#define LTFS31236D "Read block: file mark detected" -#define LTFS31283W "Cannot retrieve drive dump: failed to read buffer (%d)" -#define LTFS31224E "%s: medium is already mounted or in use" -#define LTFS31261I "Cannot get log page 0x%02x (%d) in %s" -#define LTFS31234I "Pseudo-error on %s" -#define LTFS31279W "Cannot retrieve drive dump: failed to create dump file (%d)" -#define LTFS31247E "Logical block protection Error on write" -#define LTFS31208I "IOCTL: %s %d returns %d (generic %d) %s" -#define LTFS31249I "Unrecognized space type" -#define LTFS31250I "Cannot space: count value %d of the space command is too large" -#define LTFS31220E "Invalid scsi_lbprotect option: %s" -#define LTFS31288I "Forcing drive dump" -#define LTFS31218I "Parsing log page: buffer too small, copying %zu bytes from %lx" -#define LTFS31263E "Failed to open file '%s' (%d)" -#define LTFS31266W "Cannot get EOD status: value length or partition number is wrong %d - (%d, %d)" -#define LTFS31258I "Cannot get remaining capacity: failed to parse the log page" -#define LTFS31271I "Logical block protection is enabled" -#define LTFS31214E "Error on %s: msg = NULL (%d) %s" -#define LTFS31230I "Unsupported Drive \'%s\'" -#define LTFS31392D "Backend %s %s" -#define LTFS31398D "Backend %s: (%llu, %llu) FM = %llu %s" -#define LTFS31289I "Failed to get cartridge status. The cartridge is not loaded." -#define LTFS31245W "Received low space warning (early warning) in %s" -#define LTFS31394D "Backend %s: %llu %s" -#define LTFS31212I "Cannot get sense (%d)" -#define LTFS31233I "Drive serial is %s" -#define LTFS31206D "IOCTL: sense = %02x/%02x%02x" -#define LTFS31396D "Backend %s: %zu %s" -#define LTFS31228I "Product ID is \'%s\'" -#define LTFS31256I "Invalid format mode (%d)" -#define LTFS31238D "Read block: underlength condition is detected. shortage = %d, actual = %d" -#define LTFS13015D "Unified I/O scheduler initialized." -#define LTFS13006E "Cannot initialize scheduler: failed to initialize mutex %s (%d)." -#define LTFS13026E "Write perm handling error : %s (%d)." -#define LTFS13007E "Cannot initialize scheduler: failed to initialize condition variable %s (%d)." -#define LTFS13013W "Index partition writer: failed to write data to the tape (%d)." -#define LTFS13009E "Failed to initialize mutex in scheduler private data (%d)." -#define LTFS13008E "Cannot initialize scheduler: failed to create thread %s (%d)." -#define LTFS13019E "Cannot flush: failed to write to data partition (%d)." -#define LTFS13022W "Freeing a dentry priv with outstanding write requests. This is a bug." -#define LTFS13012E "Invalid request_state received when updating the queue membership (%d)." -#define LTFS13018E "Cannot write: failed to allocate a write request." -#define LTFS13020E "Aborting full flush: flushing dentry '%s' failed (%d)." -#define LTFS13014W "Data partition writer: failed to write data to the tape (%d)." -#define LTFS13011E "Invalid back pointer to the dentry in the dentry_priv structure." -#define LTFS13025I "Get error position (%d, %d)." -#define LTFS13024I "Clean up extents and append index at index partition (%d)." -#define LTFS13021W "Failed to save index partition extents for a dentry (%d)." -#define LTFS13010E "Cannot write: failed to allocate scheduler private data (%d)." -#define LTFS13005E "Cannot initialize scheduler: failed to initialize cache manager." -#define LTFS13016D "Unified I/O scheduler uninitialized." -#define LTFS13017E "Cannot write: failed to allocate a cache block (%d)." -#define LTFS30001I "Opening a redirecting file through generic file driver (%s)." -#define LTFS30015E "Cannot read: no block at current position." -#define LTFS30154W "Cannot parse cartridge configuration file: 0-byte output." -#define LTFS30049E "Cannot allocate memory for cartridge configuration file (%d)." -#define LTFS30031E "Cannot write file marks: failed to remove current record (%d)." -#define LTFS30055E "Cannot set capacity: must issue command from partition 0, block 0." -#define LTFS30066E "Cannot write attribute: failed to write file (%d)." -#define LTFS30009E "Cannot read: check for file mark failed (%d)." -#define LTFS30074E "Cannot update EOD: failed to remove file (%d)." -#define LTFS30025E "Cannot write: failed to write file (%d)." -#define LTFS30041E "Cannot space: Unrecognized space type." -#define LTFS30039E "Cannot locate: failed to generate file name." -#define LTFS30086I "Cartridge is unsupported (%s, 0x%02x)." -#define LTFS30002E "Cannot open redirecting file (%s)." -#define LTFS30014D "Backend read: returning %zd bytes." -#define LTFS30030E "Cannot write file marks: failed to set write pass (%d)." -#define LTFS30067E "Cannot set compression: unit not ready." -#define LTFS30054E "Cannot read position: unit not ready." -#define LTFS30077E "Cannot space file marks: tried to space over EOD." -#define LTFS30155E "Cannot parse cartridge configuration file: toptag = %s." -#define LTFS30150E "Cannot parse cartridge configuration file: start document (%d)." -#define LTFS30022E "Cannot write: failed to set write pass (%d)." -#define LTFS30042E "Cannot space: failed to generate file name." -#define LTFS30198D "Backend %s: (%llu, %llu) FM = %llu." -#define LTFS30038E "Cannot locate: invalid partition %lu." -#define LTFS30050E "Cartridge configuration file is already existed as directory (%d)." -#define LTFS30037E "Cannot locate: unit not ready." -#define LTFS30195D "Backend %s." -#define LTFS30007E "Pseudo-error on %s." -#define LTFS30072E "Cannot update EOD: failed to generate file name." -#define LTFS30017E "Cannot write: unit not ready." -#define LTFS30047E "Cannot find the location pointed by redirecting file (%s)." -#define LTFS30060E "Cannot lock medium: unit not ready." -#define LTFS30057E "Cannot format: unknown format type." -#define LTFS30156E "Cannot parse cartridge configuration file: encoding = %s." -#define LTFS30027E "Cannot write: failed to update EOD (%d)." -#define LTFS30068E "Cannot find EOD: failed to generate file name." -#define LTFS30084D "Found a device (%s, %s, %s, %s)." -#define LTFS30036E "Cannot rewind: unit not ready." -#define LTFS30000I "Opening a device through generic file driver (%s)." -#define LTFS30016D "Backend write: %u bytes (at position=(%u, %llu), FMs %llu)." -#define LTFS30082I "No device directory is specified (%s)." -#define LTFS30048I "Loading a directory through generic file driver (%s)." -#define LTFS30056E "Cannot format: must issue command from partition 0, block 0." -#define LTFS30061E "Cannot unlock medium: unit not ready." -#define LTFS30157E "Cannot create xml reader for cartridge configuration file." -#define LTFS30008E "Cannot read: already at EOD." -#define LTFS30024E "Cannot write: failed to create file \'%s\' (%d)." -#define LTFS30075E "Cannot remove record: failed to make file name." -#define LTFS30040E "Cannot space: unit not ready." -#define LTFS30085I "Cartridge is read-only (%d, %s)." -#define LTFS30069E "Cannot find EOD: failed to check for file (%d)." -#define LTFS30011E "Cannot read: failed to open file (%d)." -#define LTFS30029E "Cannot write file marks: unit not ready." -#define LTFS30197D "Backend %s: (%llu, %llu)." -#define LTFS30035E "Cannot write file marks: failed to update EOD (%d)." -#define LTFS30059E "Device already reserved." -#define LTFS30005D "Backend read: %u bytes (from position=(%u, %llu), FMs %llu)." -#define LTFS30019I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." -#define LTFS30070E "Cannot find EOD: failed to update EOD (%d)." -#define LTFS30021E "Cannot write: failed to remove current record (%d)." -#define LTFS30045E "Cannot get a redirect location (%s)." -#define LTFS30051E "Cannot load cartridge configuration file (%d)." -#define LTFS30158E "Cannot parse cartridge configuration file: XML." -#define LTFS30078E "Cannot space file marks: failed to generate file name." -#define LTFS30062W "Cannot read attribute: failed to open file (%d)." -#define LTFS30010E "Cannot read: check for record failed (%d)." -#define LTFS30034E "Cannot write file marks: failed to close file (%d)." -#define LTFS30006E "Cannot read: unit not ready." -#define LTFS30058E "Cannot get remaining capacity: unit not ready." -#define LTFS30073E "Cannot update EOD: failed to create file (%d)." -#define LTFS30151E "Cannot parse cartridge configuration file: end document (%d)." -#define LTFS30046I "Dummy device shows empty (%s)." -#define LTFS30018E "Cannot write: null input buffer." -#define LTFS30083I "Scanning the devices directory (%s)." -#define LTFS30026E "Cannot write: failed to close file (%d)." -#define LTFS30063W "Cannot read attribute: failed to read file (%d)." -#define LTFS30013E "Cannot read: failed to close file (%d)." -#define LTFS30033E "Cannot write file marks: failed to create file \'%s\' (%d)." -#define LTFS30003I "Opening a directory through generic file driver (%s)." -#define LTFS30064E "Cannot write attribute: failed to generate file name." -#define LTFS30088I "Unsupported density code (0x%x, 0x%02x)." -#define LTFS30152E "Cannot parse cartridge configuration file: writer creation." -#define LTFS30023E "Cannot write: failed to generate file name." -#define LTFS30076E "Cannot remove record: failed to unlink file (%d)." -#define LTFS30043E "Cannot erase: unit not ready." -#define LTFS30199I "FILE backend options:\n -o devname= LTFS emulation directory (default=%s)\n" -#define LTFS30053E "Cannot load: failed to get write pass." -#define LTFS30080E "Cannot space file marks: tried to space over BOT." -#define LTFS30028D "Backend write file marks: %u (at position=(%u, %llu), FMs %llu)." -#define LTFS30012E "Cannot read: failed to read file (%d)." -#define LTFS30196D "Backend %s: %llu." -#define LTFS30032E "Cannot write file marks: failed to generate file name." -#define LTFS30065E "Cannot write attribute: failed to create file (%d)." -#define LTFS30153E "Cannot parse cartridge configuration file: writer fail." -#define LTFS30004E "Cannot open the device directory (%s)." -#define LTFS30020E "Cannot write: requested size (%u) exceeds maximum block size (%u)." -#define LTFS30071E "Cannot update EOD: failed to remove current record (%d)." -#define LTFS30081I "Getting the device directory (%s)." -#define LTFS30044D "Erase partition %lu." -#define LTFS30052E "Cannot load: failed to find EOD on partition %u (%d)." -#define LTFS30079E "Cannot space file marks: failed to check for file (%d)." -#define LTFS30840W "Cannot get EOD status: failed to get log page 0x%02x (%d)." -#define LTFS30812I "Cannot get the drive identifier of %s." -#define LTFS30821D "Read block: underrun in illegal length. residual = %d, actual = %d." -#define LTFS30841W "Cannot get EOD status: failed to parse the log page." -#define LTFS30802I "Cannot set timeout, The opcode = %02x (%d)." -#define LTFS30845E "Encryption feature is not supported on the drive: %d." -#define LTFS30810I "Opening a device through iokit driver (%s)." -#define LTFS30823I "Drive cleaning required." -#define LTFS30857D "Total dump data length is %lld." -#define LTFS30855I "Saving drive dump to %s." -#define LTFS30838I "Cannot parse the log page 0x%02x in %s." -#define LTFS30843E "Invalid scsi_lbprotect option: %s." -#define LTFS30827D "EOD detected (%s): ignore sense." -#define LTFS30805D "CDB %s." -#define LTFS30816I "Firmware revision is %s." -#define LTFS30829I "A long data wipe is in progress. (%d minutes passed)." -#define LTFS30806I "Unexpected CDB status: 0x%08x." -#define LTFS30826W "Received low space warning (programmable early warning) in %s." -#define LTFS30853I "Logical block protection is enabled." -#define LTFS30814I "Vendor ID is %s." -#define LTFS30825W "Received low space warning (early warning) in %s." -#define LTFS30800I "Cannot set scatter gather, The opcode = %02x (%d)." -#define LTFS30804D "CDB check condition: sense = %06x, %s." -#define LTFS30819W "A length mismatch is detected. (Act = %d, resid = %d, resid_sense = %d)." -#define LTFS30832I "Cannot get remaining capacity: get log page 0x%02x failed (%d)." -#define LTFS30837I "Cannot get log page 0x%02x (%d) in %s." -#define LTFS30859D "Transferring dump data." -#define LTFS30836I "Cannot read attribute (%d)." -#define LTFS30834I "Unexpected field in remaining capacity (0x%x)." -#define LTFS30869W "The drive is already reserved: %s (%s)" -#define LTFS30998D "Backend %s: (%llu, %llu) FM = %llu %s." -#define LTFS30867W "WWPID of reservation: x%02x%02x%02x%02x%02x%02x%02x%02x (%s)" -#define LTFS30865I "%s returns %s (%d) %s." -#define LTFS30835I "Cannot get remaining capacity: failed to parse the log page." -#define LTFS30809I "Cannot open device: inquiry failed (%d)." -#define LTFS30833I "Cannot parse remaining capacity (0x%x, %d)." -#define LTFS30844E "Encryption method of the drive is not AME but %s (0x%02X)." -#define LTFS30861W "Cannot retrieve drive dump: failed to write to dump file (%d)." -#define LTFS30868I "Retry to reserve from key registration (%s)" -#define LTFS30801I "Cannot set CDB, The opcode = %02x (%d)." -#define LTFS30999I "iokit backend options:\n" -#define LTFS30811I "Cannot open device: failed to convert devname to devnumber (%s)." -#define LTFS30860W "Cannot retrieve drive dump: failed to read buffer (%d)." -#define LTFS30803I "Failed to execute CDB, The opcode = %02x (%d)." -#define LTFS30824E "Logical block protection Error on read." -#define LTFS30839I "Unexpected parameter size for getting active CQ loss write (%d)." -#define LTFS30822D "Read block: file mark detected." -#define LTFS30820I "Read block: overrun in illegal length. residual = %d, actual = %d." -#define LTFS30846I "Pseudo-error on %s." -#define LTFS30854I "Logical block protection is disabled." -#define LTFS30870I "Failed to get the cartridge status. The cartridge is not loaded." -#define LTFS30817I "Drive serial is %s." -#define LTFS30996D "Backend %s: %llu %s." -#define LTFS30828I "Unrecognized space type." -#define LTFS30807D "%s (0x%02x) expected error %d." -#define LTFS30856W "Cannot retrieve drive dump: failed to create dump file (%d)." -#define LTFS30815I "Product ID is \'%s\'." -#define LTFS30995D "Backend %s: %zu bytes %s." -#define LTFS30847I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." -#define LTFS30997D "Backend %s: (%llu, %llu) %s." -#define LTFS30813I "Unsupported drive \'%s\' / \'%s\'." -#define LTFS30863I "Taking drive dump in buffer." -#define LTFS30994D "Backend %s: %zu %s." -#define LTFS30992D "Backend %s %s." -#define LTFS30866E "%s returns msg = NULL (%d) %s." -#define LTFS30858D "Total number of transfers is %d." -#define LTFS30842W "Cannot get EOD status: value length or partition number is wrong %d - (%d, %d)." -#define LTFS30831I "Unsupported cartridge (0x%x, 0x%x)." -#define LTFS30993D "Backend %s: %d %s." -#define LTFS30808I "%s (0x%02x) returns %d." -#define LTFS30830I "A long data wipe is in progress. %d %%." -#define LTFS30864I "Forcing drive dump." -#define LTFS30862W "Cannot retrieve drive dump: wrote %d bytes out, expected %d." -#define LTFS30818I "Reopening a device through iokit driver (%s)." -#define LTFS30598D "Backend %s: (%llu, %llu) FM = %llu %s." -#define LTFS30485D "Transfer %d: wrote %d bytes." -#define LTFS30442I "Skip retrying write() call at (%u, %llu), because the record was already written (%u, %llu)." -#define LTFS30439E "Logical block protection Error on read." -#define LTFS30478I "Saving drive dump to %s." -#define LTFS30401D "SIOC_PASS_THROUGH: no sense info: T%02x:M%02x:H%02x:D%02x %s." -#define LTFS30415W "Cannot detect lin_tape version." -#define LTFS30425I "Cannot open device \'%s\' (%d)." -#define LTFS30470E "Encryption feature is not supported on the drive: %d." -#define LTFS30465W "Cannot get EOD status: failed to parse the log page." -#define LTFS30483W "Cannot retrieve drive dump: failed to read buffer (%d)." -#define LTFS30423I "Opening a device through ibmtape driver (%s)." -#define LTFS30486W "Cannot retrieve drive dump: wrote %d bytes out, expected %d." -#define LTFS30599I "IBMTAPE backend options:\n -o devname= tape device (default=%s)\n" -#define LTFS30453W "Failed to get medium type code: medium type check is skipped." -#define LTFS30414E "Error on %s: msg = NULL (%d) %s." -#define LTFS30438D "Read block: underlength condition is detected. shortage = %d, actual = %d." -#define LTFS30462I "Cannot parse the log page 0x%02x in %s." -#define LTFS30402D "SIOC_PASS_THROUGH: sense = %02x/%02x%02x." -#define LTFS30430I "Unsupported Drive \'%s\'." -#define LTFS30443W "Enexpected current position (%d). (%u, %llu), (%u, %llu) ." -#define LTFS30408I "IOCTL: %s %d returns %d (generic %d) %s." -#define LTFS30436D "Read block: file mark detected." -#define LTFS30448D "EOD detected (%s): ignore sense." -#define LTFS30440W "Detect ENOMEM in write() call. Wait 3 secs and retry (%d)." -#define LTFS30407D "IOCTL: pos = 0x%02x%02x%02x%02x %s." -#define LTFS30592D "Backend %s %s." -#define LTFS30417E "Old lin_tape version is detected." -#define LTFS30427I "Cannot open device: inquiry failed (%d)." -#define LTFS30456I "Invalid format mode (%d)." -#define LTFS30467I "Unexpected parameter size for getting active CQ loss write (%d)." -#define LTFS30409D "IOCTL: no sense info." -#define LTFS30431I "Cannot open device: inquiry 0x%x failed (%d)." -#define LTFS30449I "Unrecognized space type." -#define LTFS30593D "Backend %s: %d %s." -#define LTFS30441I "Retrying write() call at (%u, %llu)." -#define LTFS30416I "lin_tape version is %s." -#define LTFS30484W "Cannot retrieve drive dump: failed to write to dump file (%d)." -#define LTFS30471I "Logical block protection is enabled." -#define LTFS30479W "Cannot retrieve drive dump: failed to create dump file (%d)." -#define LTFS30455I "Unsupported cartridge (0x%x, 0x%x)." -#define LTFS30400I "SIOC_PASS_THROUGH: command failed (%d) %s." -#define LTFS30464W "Cannot get EOD status: failed to get log page 0x%02x (%d)." -#define LTFS30434I "Pseudo-error on %s." -#define LTFS30418I "Parsing log page: buffer too small, copying %zu bytes from %lx." -#define LTFS30426W "Device \'%s\' must be opened in read-only mode." -#define LTFS30489I "Failed to get cartridge status. The cartridge is not loaded." -#define LTFS30458I "Cannot get remaining capacity: failed to parse the log page." -#define LTFS30429I "Vendor ID is %s." -#define LTFS30594D "Backend %s: %llu %s." -#define LTFS30446W "Received low space warning (programmable early warning) in %s." -#define LTFS30450I "Cannot space: count value %d of the space command is too large." -#define LTFS30481D "Total number of transfers is %d." -#define LTFS30405I "SIOC_PASS_THROUGH: status info: T%02x:M%02x:H%02x:D%02x %s." -#define LTFS30411D "IOCTL: %s %d expected error %d. retry the operation." -#define LTFS30461I "Cannot get log page 0x%02x (%d) in %s." -#define LTFS30469E "Encryption method of the drive is not AME but %s (0x%02X)." -#define LTFS30437I "Read block: overlength condition is detected. residual = %d, actual = %d." -#define LTFS30406D "IOCTL: sense = %02x/%02x%02x." -#define LTFS30466W "Cannot get EOD status: value length or partition number is wrong %d - (%d, %d)." -#define LTFS30482D "Transferring dump data." -#define LTFS30595D "Backend %s: %zu bytes %s." -#define LTFS30410D "IOCTL: %s %d expected error %d." -#define LTFS30457I "Cannot get remaining capacity: get log page 0x%02x failed (%d)." -#define LTFS30447E "Logical block protection Error on write." -#define LTFS30420E "Invalid scsi_lbprotect option: %s." -#define LTFS30487I "Taking drive dump in buffer." -#define LTFS30452I "A long data wipe is in progress. %d %%." -#define LTFS30463E "Failed to open file '%s' (%d)." -#define LTFS30444W "Position confirmation for write() call retry is failed (%d). (%u, %llu), (%u, %llu)." -#define LTFS30403D "SIOC_PASS_THROUGH: pos = 0x%02x%02x%02x%02x %s." -#define LTFS30596D "Backend %s: %zu %s." -#define LTFS30433I "Drive serial is %s." -#define LTFS30413I "Error on %s: %s (%d) %s." -#define LTFS30472I "Logical block protection is disabled." -#define LTFS30435I "Pseudo-error on write. Good return code, but a record to emulate a write error did not get sent to the drive." -#define LTFS30419I "Option parsing for the ibmtape backend failed (%d)." -#define LTFS30424E "%s: medium is already mounted or in use." -#define LTFS30488I "Forcing drive dump." -#define LTFS30459I "Cannot get remaining capacity: loop index error (%d)." -#define LTFS30428I "Product ID is \'%s\'." -#define LTFS30451I "A long data wipe is in progress. (%d minutes passed)." -#define LTFS30404I "SIOC_PASS_THROUGH: cannot sense data from the drive." -#define LTFS30597D "Backend %s: (%llu, %llu) %s." -#define LTFS30445W "Received low space warning (early warning) in %s." -#define LTFS30460I "Cannot read attribute (%d)." -#define LTFS30480D "Total dump data length is %lld." -#define LTFS30468I "Cannot allocate memory in %s." -#define LTFS30432I "Firmware revision is %s." -#define LTFS30412I "Cannot get sense (%d)." -#define LTFS39810W "No IP address was found. Use host name based reservation key." -#define LTFS39805W "The timeout table is not configured: SCSI OP code 0x%02x. Use the default timeout." -#define LTFS39801D "SCSI timeout (op_code 0x%02x, timeout = %d)." -#define LTFS39811W "Cannot fetch network I/F information. Use host name based reservation key. (%d)" -#define LTFS39809D "WORM cartridge is loaded." -#define LTFS39803E "CRC check failed: Len = %d, Actual CRC = %08x, Expected CRC = %08x." -#define LTFS39804D "CRC: %s ,Len = %d, CRC = %08x." -#define LTFS39813W "Drive firmware level does not correctly detect the EOD status." -#define LTFS39806D "Is medium mountable: six-character bar code %s." -#define LTFS39800W "Unsupported SCSI OP code 0x%02x." -#define LTFS39812W "Drive firmware must be updated. Upgrading to %s or later is recommended." -#define LTFS39808I "Is medium mountable: unsupported medium %s is detected." -#define LTFS39807E "Is medium mountable: invalid bar code %s." -#define LTFS39802W "Unknown SCSI OP code 0x%02x, use default timeout." -#define LTFS15506E "Failed to parse data key and/or data key identifier." -#define LTFS15508I "Key manager interface simple plug-in options:\n" -#define LTFS15507E "Cannot find data key of the data key identifier." -#define LTFS15505E "Invalid sequence error (%d,%d): %s." -#define LTFS15504E "Option parsing for the key manager interface backend failed (%d)." -#define LTFS15502E "Encryption key format violation (%s): %s." -#define LTFS15503E "Cannot find data key." -#define LTFS15501D "Simple plug-in uninitialized." -#define LTFS15500D "Simple plug-in initialized." -#define LTFS15413I " -i, --config= Use the specified configuration file (default: %s)" -#define LTFS15023I "Formatting failed." -#define LTFS15057I "Need to specify the correct data key or -f option to format this medium." -#define LTFS15012E "Failed to format the medium." -#define LTFS15055E "Unknown option '%s %s'." -#define LTFS15031E "Volume name must conform to file name rules." -#define LTFS15062E "Failed to unformat the medium due to WORM error." -#define LTFS15053E "Key manager interface backend option parsing failed." -#define LTFS15004I "LTFS volume blocksize: %lu." -#define LTFS15049I "Checking the medium (%s)." -#define LTFS15424I " --long-wipe Unformat the medium and erase any data on the tape by overwriting special data pattern.\n This operation takes over 3 hours. Once you start, you cannot interrupt it" -#define LTFS15409I " -h, --help This help" -#define LTFS15417I " -x, --fulltrace Enable full function call tracing (slow)" -#define LTFS15021W "Cannot unload backend." -#define LTFS15410I "Usage example:" -#define LTFS15059I "%s version %s." -#define LTFS15040I "Medium unformatted successfully." -#define LTFS15058E "Cannot set the tape attribute: %s." -#define LTFS15010I "Creating data partition %c on SCSI partition %d." -#define LTFS15401I "Available options are:" -#define LTFS15042I "%s." -#define LTFS15011I "Creating index partition %c on SCSI partition %d." -#define LTFS15421I " --kmi-backend= Use the specified key manager interface backend (default: %s)" -#define LTFS15411I " %s --device=%s --rules=\"%s\"" -#define LTFS15028E "Block size must be at least %d." -#define LTFS15415I " -b, --blocksize= Set the LTFS record size (default: %d)" -#define LTFS15015E "Cannot parse data placement rules (%d)." -#define LTFS15052E "Could not initialize the key manager interface plug-in. \'%s\' (%d)." -#define LTFS15038E "Failed to unformat the medium." -#define LTFS15024I "Medium formatted successfully." -#define LTFS15050E "Cannot open key manager interface backend \'%s\'." -#define LTFS15402I " -d, --device= Tape device (required)" -#define LTFS15001E "Cannot allocate LTFS volume structure." -#define LTFS15423I " -V, --version Version information" -#define LTFS15003I "Formatting device \'%s\'." -#define LTFS15007D "Device opened." -#define LTFS15406I " --no-override Disallow mount-time data placement policy changes" -#define LTFS15047E "Medium is already formatted (%d)." -#define LTFS15034E "Cannot format data placement rules (%d)." -#define LTFS15054E "Tape backend option parsing failed." -#define LTFS15048I "Need to specify -f or --force option to format this medium." -#define LTFS15020D "Closing the device." -#define LTFS15418I " -w, --wipe Restore the LTFS medium to an unpartitioned medium (format to a legacy scratch medium)" -#define LTFS15022D "Device closed." -#define LTFS15404I " -n, --volume-name= Tape volume name (empty by default)" -#define LTFS15005I "Index partition placement policy: %s." -#define LTFS15013I "Volume UUID is: %s." -#define LTFS15030E "Tape serial must contain only ASCII digits and capital letters." -#define LTFS15008E "Cannot open backend \'%s\'." -#define LTFS15009E "Cannot open device \'%s\' (%d)." -#define LTFS15046E "Unformatting was canceled by the user." -#define LTFS15056E "Failed to decrypt the medium." -#define LTFS15025D "Validating command line options." -#define LTFS15414I " -e, --backend= Use the specified tape device backend (default: %s)" -#define LTFS15420I " -f, --force Force to format medium" -#define LTFS15060E "Index criteria update is not allowed on this medium." -#define LTFS15019I "Volume capacity is %llu GB." -#define LTFS15041I "Launched by \"%s\"." -#define LTFS15412I " -p, --advanced-help Full help, including advanced options" -#define LTFS15044E "Cannot set up tape device." -#define LTFS15043I "GCC version is %s." -#define LTFS15061E "Failed to format the medium due to WORM error." -#define LTFS15051W "Cannot unload key manager interface backend." -#define LTFS15045E "Formatting was canceled by the user." -#define LTFS15029E "Tape serial must be 6 characters." -#define LTFS15416I " -c, --no-compression Disable compression on the volume" -#define LTFS15400I "Usage: %s " -#define LTFS15026E "Device name must use the \'%s\' option." -#define LTFS15014E "Cannot set policy override flag in the index (%d)." -#define LTFS15002E "Option validation failed." -#define LTFS15403I " -s, --tape-serial= Tape serial number (6 alphanumeric ASCII characters)" -#define LTFS15405I " -r, --rules= Rules for choosing files to write to the index partition.\n The syntax of the rule argument is:\n size=1M\n size=1M/name=pattern\n size=1M/name=pattern1:pattern2:pattern3\n A file is written to the index partition if it is no larger\n than the given size AND matches at least one of the name\n patterns (if specified). The size argument accepts K, M, and G\n suffixes. Name patterns might contain the special characters\n '?' (match any single character) and '*' (match zero or more\n characters)" -#define LTFS15422I " --syslogtrace Enable diagnostic output to stderr and syslog" -#define LTFS15000I "Starting mkltfs, %s version %s, log level %d." -#define LTFS15037D "Command line options are valid." -#define LTFS15419I " -k, --keep-capacity Keep the tape medium's total capacity proportion" -#define LTFS15407I " -q, --quiet Suppress progress information and general messages" -#define LTFS15039I "Unformatting failed." -#define LTFS15006D "Opening the device." -#define LTFS15408I " -t, --trace Enable function call tracing" -#define LTFS15562E "Encryption key format violation (%s): %s." -#define LTFS15563E "Cannot find data key." -#define LTFS15551D "Flat File plug-in uninitialized." -#define LTFS15550D "Flat File plug-in initialized." -#define LTFS15564E "Option parsing for the key manager interface backend failed (%d)." -#define LTFS15552E "Flat File plug-in failed to parse options." -#define LTFS15565E "Invalid sequence error (%d,%d): %s." -#define LTFS15567E "Cannot find data key of the data key identifier." -#define LTFS15553E "Flat File plug-in failed to open the file (%s, %d)." -#define LTFS15568I "Key manager interface flatfile plug-in options:\n" -#define LTFS15566E "Failed to parse data key and/or data key identifier." -#define LTFS15554E "Flat File plug-in detected key format violation." diff --git a/src/tape_drivers/freebsd/cam/.gitignore b/src/tape_drivers/freebsd/cam/.gitignore index b4630e16..2f6e991c 100644 --- a/src/tape_drivers/freebsd/cam/.gitignore +++ b/src/tape_drivers/freebsd/cam/.gitignore @@ -1,3 +1,4 @@ vendor_compat.c ibm_tape.c hp_tape.c +quantum_tape.c diff --git a/src/tape_drivers/freebsd/cam/cam_cmn.c b/src/tape_drivers/freebsd/cam/cam_cmn.c index 1b65df2e..6fef2b8e 100644 --- a/src/tape_drivers/freebsd/cam/cam_cmn.c +++ b/src/tape_drivers/freebsd/cam/cam_cmn.c @@ -775,6 +775,9 @@ int camtape_takedump_drive(void *device, bool nonforced_dump) struct camtape_data *softc = (struct camtape_data *)device; unsigned char *serial = softc->drive_serial; + if (softc->vendor != VENDOR_IBM) + return 0; + ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_TAKEDUMPDRV)); /* Make base filename */ time(&now); diff --git a/src/tape_drivers/freebsd/cam/cam_cmn.h b/src/tape_drivers/freebsd/cam/cam_cmn.h index f2e52066..5b3a1afd 100644 --- a/src/tape_drivers/freebsd/cam/cam_cmn.h +++ b/src/tape_drivers/freebsd/cam/cam_cmn.h @@ -382,6 +382,7 @@ struct camtape_data { bool loaded; /**< Is cartridge loaded? */ bool loadfailed; /**< Is load/unload failed? */ unsigned char drive_serial[255]; /**< serial number of device */ + int vendor; /**< drive vendor */ int drive_type; /**< device type */ int itd_command_size; /**< ITD sense conversion table size for commands */ struct itd_conversion_entry *itd_command; /**< ITD sense conversion table for commands */ diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 374bae7f..f8067a50 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -279,7 +279,8 @@ int camtape_open(const char *devname, void **handle) ltfsmsg(LTFS_INFO, 31229I, vendor); /* Check the drive is supportable */ - struct supported_device **cur = ibm_supported_drives; + softc->vendor = get_vendor_id(vendor); + struct supported_device **cur = get_supported_devs(softc->vendor); while(*cur) { if ((! strncmp((char*)softc->cd->inq_data.vendor, (*cur)->vendor_id, strlen((*cur)->vendor_id)) ) && @@ -294,12 +295,11 @@ int camtape_open(const char *devname, void **handle) if (drive_type != DRIVE_UNSUPPORTED) { softc->drive_type = drive_type; - /* Setup IBM tape specific parameters */ - standard_table = standard_tape_errors; - vendor_table = ibm_tape_errors; - - /* Set specific timeout value based on drive type */ - ibm_tape_init_timeout(&softc->timeouts, softc->drive_type); + /* Setup vendor specific parameters */ + init_error_table(softc->vendor, &standard_table, &vendor_table); + init_timeout(softc->vendor, &softc->timeouts, softc->drive_type); + if (!softc->timeouts) + ibm_tape_init_timeout(&softc->timeouts, softc->drive_type); } else { ltfsmsg(LTFS_INFO, 31230I, softc->cd->inq_data.product); close(softc->fd_sa); @@ -313,7 +313,7 @@ int camtape_open(const char *devname, void **handle) memcpy(softc->drive_serial, softc->cd->serial_num, softc->cd->serial_num_len); ltfsmsg(LTFS_INFO, 31232I, softc->cd->inq_data.revision); - if (! ibm_tape_is_supported_firmware(softc->drive_type, (uint8_t *)softc->cd->inq_data.revision)) { + if (! drive_has_supported_fw(softc->vendor, softc->drive_type, (uint8_t *)softc->cd->inq_data.revision)) { ltfsmsg(LTFS_INFO, 31230I, "firmware", softc->cd->inq_data.revision); close(softc->fd_sa); close_cd_pass_device(softc); @@ -1136,13 +1136,21 @@ int camtape_load(void *device, struct tc_position *pos) softc->cart_type = buf[2]; softc->density_code = buf[8]; + if (softc->vendor == VENDOR_HP) { + softc->cart_type = assume_cart_type(softc->density_code); + if (buf[2] == 0x01) + softc->is_worm = true; + } else { + softc->cart_type = buf[2]; + } + if (softc->cart_type == 0x00) { ltfsmsg(LTFS_WARN, 31253W); ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOAD)); return 0; } - rc = ibm_tape_is_supported_tape(softc->cart_type, softc->density_code, &(softc->is_worm)); + rc = is_supported_tape(softc->cart_type, softc->density_code, &(softc->is_worm)); if(rc == -LTFS_UNSUPPORTED_MEDIUM) ltfsmsg(LTFS_INFO, 31255I, softc->cart_type, softc->density_code); @@ -2363,13 +2371,18 @@ int camtape_set_default(void *device) } /* set logical block protection */ - if (global_data.crc_checking) { - ltfsmsg(LTFS_DEBUG, 31392D, __FUNCTION__, "Setting LBP"); - rc = camtape_set_lbp(device, true); + if (softc->vendor == VENDOR_IBM) { + if (global_data.crc_checking) { + ltfsmsg(LTFS_DEBUG, 31392D, __FUNCTION__, "Setting LBP"); + rc = camtape_set_lbp(device, true); + } else { + ltfsmsg(LTFS_DEBUG, 31392D, __FUNCTION__, "Resetting LBP"); + rc = camtape_set_lbp(device, false); + } } else { - ltfsmsg(LTFS_DEBUG, 31392D, __FUNCTION__, "Resetting LBP"); - rc = camtape_set_lbp(device, false); + rc = DEVICE_GOOD; } + if (rc != DEVICE_GOOD) goto bailout; diff --git a/src/tape_drivers/linux/sg/.gitignore b/src/tape_drivers/linux/sg/.gitignore index ce4d0b0c..0907b2ce 100644 --- a/src/tape_drivers/linux/sg/.gitignore +++ b/src/tape_drivers/linux/sg/.gitignore @@ -1,4 +1,5 @@ vendor_compat.c ibm_tape.c hp_tape.c +quantum_tape.c open_factor.c diff --git a/src/tape_drivers/linux/sg/quantum_tape.c b/src/tape_drivers/linux/sg/quantum_tape.c deleted file mode 120000 index 38c99925..00000000 --- a/src/tape_drivers/linux/sg/quantum_tape.c +++ /dev/null @@ -1 +0,0 @@ -../../quantum_tape.c \ No newline at end of file From b2d829885030fcaa29ac43f743599c241ae788b1 Mon Sep 17 00:00:00 2001 From: Michael McKendricks <38221621+softloft38p-michael@users.noreply.github.com> Date: Tue, 2 Feb 2021 17:10:59 -0800 Subject: [PATCH 015/121] Handle the no xattr destination on ltfs_ordered_copy Treat errno 95 on ltfs_ordered_copy to support destination file system which doesn't support xattr. Fix #236, #243 --- src/utils/ltfs_ordered_copy | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/src/utils/ltfs_ordered_copy b/src/utils/ltfs_ordered_copy index 19b5356a..8c58d616 100755 --- a/src/utils/ltfs_ordered_copy +++ b/src/utils/ltfs_ordered_copy @@ -267,7 +267,7 @@ logger_debug = 6 parser = argparse.ArgumentParser(description = 'Copy files from source to destination with LTFS order optimization') parser.add_argument('SOURCE', help='source files', nargs='*') parser.add_argument('DEST', help='destination', nargs='?') -parser.add_argument('-p', help='preserve attributes with shutil.copy2()', action='store_true') +parser.add_argument('-p', help='preserve attributes with shutil.copy2(). Ignored if destination does not support xattr', action='store_true') parser.add_argument('-r', '--recursive', help='copy directories recursively', action='store_true') parser.add_argument('-t', '--target-directory', help='copy all SOURCE arguments into TARGET_DIRECTORY') #parser.add_argument('-z', '--zero', help='handle NULL delimited source list (assume input \'find -print0\')', action='store_true') @@ -359,9 +359,12 @@ try: else: logger.info("Destination {0} is not LTFS\n".format(args.DEST)) except IOError as e: - if e.errno != 61 and e.errno != 93: # Not ENODATA, ENOATTR + if e.errno != 61 and e.errno != 93 and e.errno != 95: # Not ENODATA, ENOATTR, EOPNOTSUPP logger.error('Check destination (I/O):' + str(e)) exit(2) + if e.errno == 95 and args.p: + logger.warning("{0} does not support xattr. Cannot use -p. Attributes will not be preserved during copy.".format(args.DEST)) + args.p = False logger.warning("Destination {0} is not LTFS".format(args.DEST)) except Exception as e: logger.error('Check destination:' + str(e)) From f2769f20688ac90396f1f5d7f5473dfebb9de98e Mon Sep 17 00:00:00 2001 From: Michael McKendricks <38221621+softloft38p-michael@users.noreply.github.com> Date: Tue, 2 Feb 2021 17:12:46 -0800 Subject: [PATCH 016/121] Honor `--keep-tree` for single file copies (#244) --- src/utils/ltfs_ordered_copy | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/utils/ltfs_ordered_copy b/src/utils/ltfs_ordered_copy index 8c58d616..10f14db8 100755 --- a/src/utils/ltfs_ordered_copy +++ b/src/utils/ltfs_ordered_copy @@ -338,6 +338,12 @@ if args.recursive == False and len(args.SOURCE) == 1: logger.log(NOTSET + 1, 'Single file copy') if os.path.isfile(args.SOURCE[0]): try: + if len(args.keep_tree): + dst = args.DEST + '/' + args.SOURCE[0][len(args.keep_tree):] + args.DEST = os.path.normpath(dst) + (new_d, t) = os.path.split(dst) + if not os.path.exists(new_d): + os.makedirs(new_d) shutil.copy(args.SOURCE[0], args.DEST) except Exception as e: logger.error(str(e)) From d874e20ee27abc7fe5806b5d28a14620d29eaff6 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 8 Feb 2021 13:58:06 +0900 Subject: [PATCH 017/121] Support another Quantum drive (#247) --- messages/tape_iokit/root.txt | 1 + src/tape_drivers/freebsd/cam/cam_tc.c | 11 ++++++++--- src/tape_drivers/linux/sg/sg_tape.c | 8 +++++++- src/tape_drivers/osx/iokit/iokit_tape.c | 14 +++++++++++++- src/tape_drivers/quantum_tape.c | 1 + 5 files changed, 30 insertions(+), 5 deletions(-) diff --git a/messages/tape_iokit/root.txt b/messages/tape_iokit/root.txt index 91ec17fc..ff2c152f 100644 --- a/messages/tape_iokit/root.txt +++ b/messages/tape_iokit/root.txt @@ -115,6 +115,7 @@ root:table { 30868I:string { "Retry to reserve from key registration (%s)" } 30869W:string { "The drive is already reserved: %s (%s)" } 30870I:string { "Failed to get the cartridge status. The cartridge is not loaded." } + 30871W:string { "Failed to get medium type code: medium type check is skipped." } 30992D:string { "Backend %s %s." } 30993D:string { "Backend %s: %d %s." } diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index f8067a50..e4afee7a 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -1133,14 +1133,19 @@ int camtape_load(void *device, struct tc_position *pos) softc->force_readperm = DEFAULT_READPERM; softc->write_counter = 0; softc->read_counter = 0; - softc->cart_type = buf[2]; softc->density_code = buf[8]; - if (softc->vendor == VENDOR_HP) { - softc->cart_type = assume_cart_type(softc->density_code); + if (buf[2] == 0x00 || buf[2] == 0x01) { + /* + * Non-IBM drive doesn't have cartridge type so need to assume from density code. + */ + softc->cart_type = assume_cart_type(priv->density_code); if (buf[2] == 0x01) softc->is_worm = true; } else { + /* + * IBM drive haves cartridge type in buf[2] like TC_MP_LTO5D_CART. + */ softc->cart_type = buf[2]; } diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index e72eb2db..b849692f 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -2595,11 +2595,17 @@ int sg_load(void *device, struct tc_position *pos) priv->density_code = buf[8]; - if (priv->vendor == VENDOR_HP) { + if (buf[2] == 0x00 || buf[2] == 0x01) { + /* + * Non-IBM drive doesn't have cartridge type so need to assume from density code. + */ priv->cart_type = assume_cart_type(priv->density_code); if (buf[2] == 0x01) priv->is_worm = true; } else { + /* + * IBM drive haves cartridge type in buf[2] like TC_MP_LTO5D_CART. + */ priv->cart_type = buf[2]; } diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index fd230675..98623c74 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -1980,14 +1980,26 @@ int iokit_load(void *device, struct tc_position *pos) priv->density_code = buf[8]; - if (priv->vendor == VENDOR_HP) { + if (buf[2] == 0x00 || buf[2] == 0x01) { + /* + * Non-IBM drive doesn't have cartridge type so need to assume from density code. + */ priv->cart_type = assume_cart_type(priv->density_code); if (buf[2] == 0x01) priv->is_worm = true; } else { + /* + * IBM drive haves cartridge type in buf[2] like TC_MP_LTO5D_CART. + */ priv->cart_type = buf[2]; } + if (priv->cart_type == 0x00) { + ltfsmsg(LTFS_WARN, 30871W); + ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOAD)); + return 0; + } + ret = is_supported_tape(priv->cart_type, priv->density_code, &(priv->is_worm)); if(ret == -LTFS_UNSUPPORTED_MEDIUM) ltfsmsg(LTFS_INFO, 30831I, priv->cart_type, priv->density_code); diff --git a/src/tape_drivers/quantum_tape.c b/src/tape_drivers/quantum_tape.c index 9f583ac3..504ee8d7 100644 --- a/src/tape_drivers/quantum_tape.c +++ b/src/tape_drivers/quantum_tape.c @@ -67,6 +67,7 @@ struct supported_device *quantum_supported_drives[] = { TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH6", DRIVE_LTO6_HH, "[ULTRIUM-HH6]" ), /* QUANTUM Ultrium Gen 6 Half-High */ TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH7", DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* QUANTUM Ultrium Gen 7 Half-High */ TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH8", DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* QUANTUM Ultrium Gen 8 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 5", DRIVE_LTO5_HH, "[ULTRIUM-5]" ), /* Another QUANTUM Ultrium Gen 5 Half-High */ /* End of supported_devices */ NULL }; From 3b5acac2736b4e6e1a2da30cfbdfa87de3884c70 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 8 Feb 2021 14:07:13 +0900 Subject: [PATCH 018/121] Fix remaining capacity handling of HP LTO6 (#248) --- src/tape_drivers/freebsd/cam/cam_tc.c | 4 +++- src/tape_drivers/linux/sg/sg_tape.c | 4 +++- src/tape_drivers/osx/iokit/iokit_tape.c | 4 +++- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index e4afee7a..e3333b15 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -1771,7 +1771,9 @@ int camtape_remaining_capacity(void *device, struct tc_remaining_cap *cap) int rc; ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_REMAINCAP)); - if (IS_LTO(softc->drive_type) && (DRIVE_GEN(softc->drive_type) == 0x05)) { + if ((IS_LTO(softc->drive_type) && (DRIVE_GEN(softc->drive_type) == 0x05)) || + (softc->vendor == VENDOR_HP && IS_LTO(softc->drive_type) && (DRIVE_GEN(softc->drive_type) == 0x06))) { + /* Issue LogPage 0x31 */ rc = camtape_logsense(device, LOG_TAPECAPACITY, logdata, LOGSENSEPAGE); if (rc) { diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index b849692f..f8c736da 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -2879,7 +2879,9 @@ int sg_remaining_capacity(void *device, struct tc_remaining_cap *cap) memset(buffer, 0, LOGSENSEPAGE); - if (IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) { + if ((IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) || + (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))) { + /* Use LogPage 0x31 */ ret = sg_logsense(device, (uint8_t)LOG_TAPECAPACITY, (void *)buffer, LOGSENSEPAGE); if(ret < 0) diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index 98623c74..43854e97 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -2228,7 +2228,9 @@ int iokit_remaining_capacity(void *device, struct tc_remaining_cap *cap) memset(&buffer, 0, LOGSENSEPAGE); - if (IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) { + if ((IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) || + (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))) { + /* Use LogPage 0x31 */ ret = iokit_logsense(device, (uint8_t)LOG_TAPECAPACITY, (void *)buffer, LOGSENSEPAGE); if(ret < 0) From aacc20cc5bb174f9748d4a4c336c7e2fd4a4b1cf Mon Sep 17 00:00:00 2001 From: Michael McKendricks <38221621+softloft38p-michael@users.noreply.github.com> Date: Wed, 10 Feb 2021 18:37:26 -0800 Subject: [PATCH 019/121] Document required reboot for modprobe.d settings to take effect. Fixes #249 (#250) --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index f8b30346..123247ce 100644 --- a/README.md +++ b/README.md @@ -135,6 +135,8 @@ In RHEL7, you can put following file as `/etc/modprobe.d/sg.conf`. options sg def_reserved_size=1048576 ``` +A reboot is required for these settings to take effect. + You can check current configuration of sg driver to see the file `/proc/scsi/sg/debug` like ``` @@ -164,6 +166,8 @@ In RHEL7, you can put following file as `/etc/modprobe.d/sg.conf`. options sg allow_dio=1 ``` +A reboot is required for these settings to take effect. + At this time, we know following HBA's works correctly. - QLogic 8Gb FC HBAs @@ -181,6 +185,8 @@ In the lpfc driver (for Emulex Fibre HBAs), the table size of the scatter-gather options lpfc lpfc_sg_seg_cnt=256 ``` +A reboot is required for these settings to take effect. + In some versions of the lpfc driver (for Emulex Fibre HBAs), the table size of the scatter-gather cannot be changed correctly. You can check the value is changed or not in `sg_tablesize` value in `/proc/scsi/sg/debug`. If you don't have a correct value (256 or greater) in `sg_tablesize`, removing `allow_dio=1` configuration of the sg driver is strongly recommended. ##### Note for buggy HBAs From 30829ef123c9ffe00c38e832f248483ca2ce1658 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 26 Feb 2021 19:56:21 +0900 Subject: [PATCH 020/121] Fetch timeout values from drive if available (#253) This change also includes following changes. - Remove warning on RHEL8 - Enhance message checker to support RHEL8 - Improve auto build checker in macOS and RHEL8 (use message checker) - Fix unknown messages --- .github/workflows/build-centos8.yml | 2 +- .travis.yml | 2 +- configure.ac | 7 +- messages/print_error_messages.py | 76 +++++----- messages/tape_iokit/root.txt | 4 +- messages/tape_linux_sg/root.txt | 1 + src/libltfs/arch/time_internal.h | 13 +- src/libltfs/ltfs.c | 4 +- src/libltfs/tape.c | 5 +- src/ltfs_fuse.c | 10 +- src/tape_drivers/generic/file/filedebug_tc.c | 7 +- src/tape_drivers/generic/itdtimg/itdtimg_tc.c | 6 +- src/tape_drivers/ibm_tape.c | 96 ++++++------- src/tape_drivers/linux/sg/sg_tape.c | 121 +++++++++++++--- src/tape_drivers/osx/iokit/iokit_tape.c | 130 ++++++++++++++---- src/tape_drivers/tape_drivers.h | 9 +- src/tape_drivers/vendor_compat.c | 33 +++++ src/tape_drivers/vendor_compat.h | 1 + 18 files changed, 380 insertions(+), 147 deletions(-) diff --git a/.github/workflows/build-centos8.yml b/.github/workflows/build-centos8.yml index 727896e1..85a3c13a 100644 --- a/.github/workflows/build-centos8.yml +++ b/.github/workflows/build-centos8.yml @@ -12,6 +12,6 @@ jobs: uses: actions/checkout@v1 - name: Build LTFS id: build - uses: LinearTapeFileSystem/CentOS8-Build@v1.0 + uses: LinearTapeFileSystem/CentOS8-Build@v1.3 with: destination: '/tmp/ltfs' diff --git a/.travis.yml b/.travis.yml index 1a78dc40..d4f7be5e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -50,7 +50,7 @@ before_install: - if [[ "$TRAVIS_DIST" == "bionic" ]]; then sudo apt-get install -y libuuid1 uuid-dev ; fi script: - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then ./autogen.sh && LDFLAGS="-framework CoreFoundation -framework IOKit" ./configure --enable-icu-6x --disable-snmp && make ; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then ./autogen.sh && LDFLAGS="-framework CoreFoundation -framework IOKit" ./configure --enable-icu-6x --disable-snmp --enable-warning-as-error --enable-message-checker && make ; fi - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then ./autogen.sh && ./configure --prefix=/tmp && make && make install ; fi - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then cd /tmp && git clone https://github.com/LinearTapeFileSystem/ltfs-backends.git && cd ltfs-backends ; fi - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:/tmp/lib/pkgconfig ; fi diff --git a/configure.ac b/configure.ac index b8a2fee0..b61ac3cb 100644 --- a/configure.ac +++ b/configure.ac @@ -483,7 +483,7 @@ fi if test "x$warning_as_error" = "xyes" then - AM_CFLAGS="${AM_CFLAGS} -Werror" + AM_CFLAGS="${AM_CFLAGS} -Werror -Wno-deprecated-declarations" fi if test "x$use_new_locking" = "xyes" @@ -519,6 +519,11 @@ then SSE42=yes fi + if test ${GCC_VERSION_MAJOR} -ge 8 -a ${GCC_VERSION_MINOR} -ge 0 -a "x$warning_as_error" = "xyes" + then + AM_CFLAGS="${AM_CFLAGS} -Wno-stringop-truncation" + fi + if test "x${SSE42}" = "xyes" then case x"$target_cpu" in diff --git a/messages/print_error_messages.py b/messages/print_error_messages.py index fc5dc900..019fa820 100755 --- a/messages/print_error_messages.py +++ b/messages/print_error_messages.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python2.7 +#!/usr/bin/env python import os import re @@ -9,45 +9,45 @@ # List the messages defined in the message bundles for d, dirs, files in os.walk('.'): - if d == 'messages': - continue - if d == "messages/internal_error": - continue - msg_list = dict() - for f in files: - start_id = 0 - end_id = 1000000 - if re.search(r'\.txt$', f): - with file(os.path.join(d, f), 'r') as fd: - linenum = 1 - for line in fd: - m = re.search(r'start_id:int\s*{\s*(?P[0-9]+)\s*}', line) - if m is not None: - start_id = int(m.group('val')) - m = re.search(r'end_id:int\s*{\s*(?P[0-9]+)\s*}', line) - if m is not None: - end_id = int(m.group('val')) - if end_id < start_id: - print 'Warning: strange message ID range (%d-%d) in %s' % ( - start_id, end_id, os.path.join(d, f)) + if d == 'messages': + continue + if d == "messages/internal_error": + continue + msg_list = dict() + for f in files: + start_id = 0 + end_id = 1000000 + if re.search(r'\.txt$', f): + fd = open(os.path.join(d, f), 'r') + linenum = 1 + for line in fd: + m = re.search(r'start_id:int\s*{\s*(?P[0-9]+)\s*}', line) + if m is not None: + start_id = int(m.group('val')) + m = re.search(r'end_id:int\s*{\s*(?P[0-9]+)\s*}', line) + if m is not None: + end_id = int(m.group('val')) + if end_id < start_id: + print('Warning: strange message ID range (%d-%d) in %s' % ( + start_id, end_id, os.path.join(d, f))) - m = re.search(r'^(?P.*)//', line) - if m is not None: - m = re.search(re_msgid_bundle, m.group('val')) - else: - m = re.search(re_msgid_bundle, line) - if m is not None: - val = int(m.group('val')) - if val < start_id or val > end_id: - print 'Message ID %s out of range (%d-%d) at %s:%d' % ( - m.group('id'), start_id, end_id, os.path.join(d, f), linenum) - else: - msg_list[m.group('id')] = m.group('msg') - linenum += 1 - if len(msg_list) > 0: - msg_ids[os.path.basename(d)] = msg_list + m = re.search(r'^(?P.*)//', line) + if m is not None: + m = re.search(re_msgid_bundle, m.group('val')) + else: + m = re.search(re_msgid_bundle, line) + if m is not None: + val = int(m.group('val')) + if val < start_id or val > end_id: + print('Message ID %s out of range (%d-%d) at %s:%d' % ( + m.group('id'), start_id, end_id, os.path.join(d, f), linenum)) + else: + msg_list[m.group('id')] = m.group('msg') + linenum += 1 + if len(msg_list) > 0: + msg_ids[os.path.basename(d)] = msg_list # Find unused and nonexistent message IDs for module in msg_ids: for id in msg_ids[module]: - print "#define LTFS%s %s" % (id, msg_ids[module][id]) + print("#define LTFS%s %s" % (id, msg_ids[module][id])) diff --git a/messages/tape_iokit/root.txt b/messages/tape_iokit/root.txt index ff2c152f..699a0fc7 100644 --- a/messages/tape_iokit/root.txt +++ b/messages/tape_iokit/root.txt @@ -116,6 +116,7 @@ root:table { 30869W:string { "The drive is already reserved: %s (%s)" } 30870I:string { "Failed to get the cartridge status. The cartridge is not loaded." } 30871W:string { "Failed to get medium type code: medium type check is skipped." } + 30872I:string { "Setting up timeout values from %s." } 30992D:string { "Backend %s %s." } 30993D:string { "Backend %s: %d %s." } @@ -124,8 +125,7 @@ root:table { 30996D:string { "Backend %s: %llu %s." } 30997D:string { "Backend %s: (%llu, %llu) %s." } 30998D:string { "Backend %s: (%llu, %llu) FM = %llu %s." } - 30999I:string { "iokit backend options:\n" - " -o devname= tape device (default=%s)\n" + 30999I:string { "iokit backend options:\n -o devname= tape device (default=%s)\n" " -o scsi_lbprotect= enable drive logical block protection (default=off)\n." } } } diff --git a/messages/tape_linux_sg/root.txt b/messages/tape_linux_sg/root.txt index 9698160b..9e2047aa 100644 --- a/messages/tape_linux_sg/root.txt +++ b/messages/tape_linux_sg/root.txt @@ -133,6 +133,7 @@ root:table { 30291I:string { "Changer %s was reserved from this node and can be reserved from the current path." } 30292I:string { "Changer %s was reserved from this node but failed to reserve from the current path." } 30293I:string { "Changer %s was reserved from another node (%s)." } + 30294I:string { "Setting up timeout values from %s." } 30392D:string { "Backend %s %s." } 30393D:string { "Backend %s: %d %s." } diff --git a/src/libltfs/arch/time_internal.h b/src/libltfs/arch/time_internal.h index 859c9450..b84d1929 100644 --- a/src/libltfs/arch/time_internal.h +++ b/src/libltfs/arch/time_internal.h @@ -56,12 +56,19 @@ extern "C" { #ifdef mingw_PLATFORM #include "libltfs/arch/win/win_util.h" +#elif defined(__APPLE__) +#include +typedef time_t ltfs_time_t; +struct ltfs_timespec { + ltfs_time_t tv_sec; + long tv_nsec; +}; #else #include -typedef int64_t ltfs_time_t; +typedef int64_t ltfs_time_t; struct ltfs_timespec { - ltfs_time_t tv_sec; - long tv_nsec; + ltfs_time_t tv_sec; + long tv_nsec; }; #endif diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 10a1bd39..8c3a7855 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -2285,7 +2285,7 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) return ret; } - if (vol->label->barcode[0] == ' ' || vol->label->barcode == NULL) + if (vol->label->barcode[0] == ' ' || vol->label->barcode[0] == '\0') bc_print = LTFS_NO_BARCODE; else bc_print = vol->label->barcode; @@ -3398,7 +3398,7 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) releaseread_mrsw(&vol->lock); if (dirty) { - if (vol->label->barcode[0] == ' ' || vol->label->barcode == NULL) + if (vol->label->barcode[0] == ' ' || vol->label->barcode[0] == '\0') bc_print = LTFS_NO_BARCODE; else bc_print = vol->label->barcode; diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index d6ae97c9..8fcab456 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -1819,9 +1819,8 @@ int tape_get_media_pool_info(struct ltfs_volume *vol, char **media_name, char ** CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); - if (vol->t_attr->media_pool) { - len = strlen(vol->t_attr->media_pool); - } + vol->t_attr->media_pool[TC_MAM_MEDIA_POOL_SIZE] = '\0'; /* Add a sentinel */ + len = strlen(vol->t_attr->media_pool); if (len == 0) return -1; diff --git a/src/ltfs_fuse.c b/src/ltfs_fuse.c index f26501d2..db4d3a5c 100644 --- a/src/ltfs_fuse.c +++ b/src/ltfs_fuse.c @@ -329,17 +329,21 @@ int ltfs_fuse_statfs(const char *path, struct statvfs *buf) stats->f_bfree = blockstat.remaining_dp; /* Remaining tape capacity */ stats->f_bavail = stats->f_bfree; /* Blocks available for normal user (ignored) */ - stats->f_files = UINT64_MAX; - stats->f_ffree = UINT64_MAX - ltfs_get_file_count(priv->data); +#ifdef __APPLE__ + stats->f_files = UINT_MAX; + stats->f_ffree = UINT_MAX - ltfs_get_file_count(priv->data); memcpy(buf, stats, sizeof(struct statvfs)); -#ifdef __APPLE__ /* With MacFUSE, we use an f_frsize not equal to the file system block size. * Need to adjust the block counts so they're in units of the reported f_frsize. */ double scale = ltfs_get_blocksize(priv->data) / (double)stats->f_frsize; buf->f_blocks *= scale; buf->f_bfree *= scale; buf->f_bavail *= scale; +#else + stats->f_files = UINT64_MAX; + stats->f_ffree = UINT64_MAX - ltfs_get_file_count(priv->data); + memcpy(buf, stats, sizeof(struct statvfs)); #endif /* __APPLE__ */ ltfs_request_trace(FUSE_REQ_EXIT(REQ_STATFS), 0, 0); diff --git a/src/tape_drivers/generic/file/filedebug_tc.c b/src/tape_drivers/generic/file/filedebug_tc.c index 1909d74b..f08418a5 100644 --- a/src/tape_drivers/generic/file/filedebug_tc.c +++ b/src/tape_drivers/generic/file/filedebug_tc.c @@ -291,7 +291,12 @@ static void emulate_seek_wait(struct filedebug_data *state, struct tc_position * dest->block % blocks_per_wrap : blocks_per_wrap - (dest->block % blocks_per_wrap); - uint64_t distance = llabs(target_dist_from_bot - current_dist_from_bot); + uint64_t distance; + if (target_dist_from_bot > current_dist_from_bot) + distance = target_dist_from_bot - current_dist_from_bot; + else + distance = current_dist_from_bot - target_dist_from_bot; + float cost = ((float) state->conf.eot_to_bot_sec / blocks_per_wrap) * (distance-1.0); time_t delay_us = 0; diff --git a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c index d739621e..dad017d7 100644 --- a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c +++ b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c @@ -1351,9 +1351,9 @@ int itdtimage_get_device_list(struct tc_drive_info *buf, int count) if (buf && deventries < count) { snprintf(buf[deventries].name, TAPE_DEVNAME_LEN_MAX, "%s/%s", devdir, entry->d_name); - snprintf(buf[deventries].vendor, TAPE_VENDOR_NAME_LEN_MAX, "DUMMY"); - snprintf(buf[deventries].model, TAPE_MODEL_NAME_LEN_MAX, "DUMMYDEV"); - snprintf(buf[deventries].serial_number, TAPE_SERIAL_LEN_MAX, "%s", &(entry->d_name[strlen(DRIVE_FILE_PREFIX)])); + strncpy(buf[deventries].vendor, "DUMMY", TAPE_VENDOR_NAME_LEN_MAX); + strncpy(buf[deventries].model, "DUMMYDEV", TAPE_MODEL_NAME_LEN_MAX); + strncpy(buf[deventries].serial_number, &(entry->d_name[strlen(DRIVE_FILE_PREFIX)]), TAPE_SERIAL_LEN_MAX); ltfsmsg(LTFS_DEBUG, 31030D, buf[deventries].name, buf[deventries].vendor, buf[deventries].model, buf[deventries].serial_number); } diff --git a/src/tape_drivers/ibm_tape.c b/src/tape_drivers/ibm_tape.c index 88d7f576..fe7da9c6 100644 --- a/src/tape_drivers/ibm_tape.c +++ b/src/tape_drivers/ibm_tape.c @@ -500,8 +500,8 @@ static struct _timeout_tape timeout_lto6[] = { { ERASE, 24600 }, { FORMAT_MEDIUM, 3000 }, { LOAD_UNLOAD, 780 }, - { LOCATE10, 2040 }, - { LOCATE16, 2040 }, + { LOCATE10, 2940 }, + { LOCATE16, 2940 }, { READ, 1500 }, { READ_BUFFER, 480 }, { REWIND, 600 }, @@ -519,7 +519,7 @@ static struct _timeout_tape timeout_lto6[] = { static struct _timeout_tape timeout_lto7[] = { { ERASE, 27540 }, { FORMAT_MEDIUM, 3000 }, - { LOAD_UNLOAD, 780 }, + { LOAD_UNLOAD, 960 }, { LOCATE10, 2880 }, { LOCATE16, 2880 }, { READ, 2280 }, @@ -537,9 +537,9 @@ static struct _timeout_tape timeout_lto7[] = { }; static struct _timeout_tape timeout_lto8[] = { - { ERASE, 46380 }, + { ERASE, 54896 }, { FORMAT_MEDIUM, 3000 }, - { LOAD_UNLOAD, 780 }, + { LOAD_UNLOAD, 960 }, { LOCATE10, 2880 }, { LOCATE16, 2880 }, { READ, 2280 }, @@ -549,7 +549,7 @@ static struct _timeout_tape timeout_lto8[] = { { SET_CAPACITY, 780 }, { SPACE6, 2880 }, { SPACE16, 2880 }, - { VERIFY, 28860 }, + { VERIFY, 47700 }, { WRITE, 1500 }, { WRITE_BUFFER, 540 }, { WRITE_FILEMARKS6, 1620 }, @@ -557,19 +557,19 @@ static struct _timeout_tape timeout_lto8[] = { }; static struct _timeout_tape timeout_lto9[] = { - { ERASE, 46380 }, + { ERASE, 74341 }, { FORMAT_MEDIUM, 3000 }, - { LOAD_UNLOAD, 780 }, - { LOCATE10, 2880 }, - { LOCATE16, 2880 }, - { READ, 2280 }, + { LOAD_UNLOAD, 960 }, + { LOCATE10, 2940 }, + { LOCATE16, 2940 }, + { READ, 2340 }, { READ_BUFFER, 480 }, { REWIND, 600 }, { SEND_DIAGNOSTIC, 1980 }, { SET_CAPACITY, 780 }, - { SPACE6, 2880 }, - { SPACE16, 2880 }, - { VERIFY, 28860 }, + { SPACE6, 2940 }, + { SPACE16, 2940 }, + { VERIFY, 63300 }, { WRITE, 1500 }, { WRITE_BUFFER, 540 }, { WRITE_FILEMARKS6, 1620 }, @@ -619,12 +619,12 @@ static struct _timeout_tape timeout_lto6_hh[] = { static struct _timeout_tape timeout_lto7_hh[] = { { ERASE, 27540 }, { FORMAT_MEDIUM, 3240 }, - { LOAD_UNLOAD, 840 }, + { LOAD_UNLOAD, 960 }, { LOCATE10, 2940 }, { LOCATE16, 2940 }, { READ, 2340 }, { READ_BUFFER, 480 }, - { REWIND, 660 }, + { REWIND, 600 }, { SEND_DIAGNOSTIC, 2040 }, { SET_CAPACITY, 960 }, { SPACE6, 2940 }, @@ -637,42 +637,42 @@ static struct _timeout_tape timeout_lto7_hh[] = { }; static struct _timeout_tape timeout_lto8_hh[] = { - { ERASE, 46380 }, - { FORMAT_MEDIUM, 3240 }, - { LOAD_UNLOAD, 840 }, - { LOCATE10, 2940 }, - { LOCATE16, 2940 }, - { READ, 2340 }, - { READ_BUFFER, 480 }, - { REWIND, 660 }, - { SEND_DIAGNOSTIC, 2040 }, - { SET_CAPACITY, 960 }, - { SPACE6, 2940 }, - { SPACE16, 2940 }, - { VERIFY, 47700 }, - { WRITE, 1560 }, - { WRITE_BUFFER, 540 }, - { WRITE_FILEMARKS6, 1680 }, + { ERASE, 121448 }, + { FORMAT_MEDIUM, 3240 }, + { LOAD_UNLOAD, 960 }, + { LOCATE10, 2940 }, + { LOCATE16, 2940 }, + { READ, 2340 }, + { READ_BUFFER, 480 }, + { REWIND, 600 }, + { SEND_DIAGNOSTIC, 2040 }, + { SET_CAPACITY, 960 }, + { SPACE6, 2940 }, + { SPACE16, 2940 }, + { VERIFY, 54360 }, + { WRITE, 1560 }, + { WRITE_BUFFER, 540 }, + { WRITE_FILEMARKS6, 1680 }, {-1, -1} }; static struct _timeout_tape timeout_lto9_hh[] = { - { ERASE, 46380 }, - { FORMAT_MEDIUM, 3240 }, - { LOAD_UNLOAD, 840 }, - { LOCATE10, 2940 }, - { LOCATE16, 2940 }, - { READ, 2340 }, - { READ_BUFFER, 480 }, - { REWIND, 660 }, - { SEND_DIAGNOSTIC, 2040 }, - { SET_CAPACITY, 960 }, - { SPACE6, 2940 }, - { SPACE16, 2940 }, - { VERIFY, 47700 }, - { WRITE, 1560 }, - { WRITE_BUFFER, 540 }, - { WRITE_FILEMARKS6, 1680 }, + { ERASE, 166370 }, + { FORMAT_MEDIUM, 3240 }, + { LOAD_UNLOAD, 960 }, + { LOCATE10, 2940 }, + { LOCATE16, 2940 }, + { READ, 2340 }, + { READ_BUFFER, 480 }, + { REWIND, 600 }, + { SEND_DIAGNOSTIC, 2040 }, + { SET_CAPACITY, 960 }, + { SPACE6, 2940 }, + { SPACE16, 2940 }, + { VERIFY, 63300 }, + { WRITE, 1560 }, + { WRITE_BUFFER, 540 }, + { WRITE_FILEMARKS6, 1680 }, {-1, -1} }; diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index f8c736da..0f2f9de6 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -540,12 +540,12 @@ static int _raw_open(struct sg_data *priv) ltfsmsg(LTFS_INFO, 30214I, id_data.product_rev); ltfsmsg(LTFS_INFO, 30215I, priv->drive_serial); - snprintf(priv->info.name, TAPE_DEVNAME_LEN_MAX + 1, "%s", priv->devname); - snprintf(priv->info.vendor, TAPE_VENDOR_NAME_LEN_MAX + 1, "%s", id_data.vendor_id); - snprintf(priv->info.model, TAPE_MODEL_NAME_LEN_MAX + 1, "%s", id_data.product_id); - snprintf(priv->info.serial_number, TAPE_SERIAL_LEN_MAX + 1, "%s", id_data.unit_serial); - snprintf(priv->info.product_rev, PRODUCT_REV_LENGTH + 1, "%s", id_data.product_rev); - snprintf(priv->info.product_name, PRODUCT_NAME_LENGTH + 1, "%s", _generate_product_name(id_data.product_id)); + strncpy(priv->info.name, priv->devname, TAPE_DEVNAME_LEN_MAX + 1); + strncpy(priv->info.vendor, id_data.vendor_id, TAPE_VENDOR_NAME_LEN_MAX + 1); + strncpy(priv->info.model, id_data.product_id, TAPE_MODEL_NAME_LEN_MAX + 1); + strncpy(priv->info.serial_number, id_data.unit_serial, TAPE_SERIAL_LEN_MAX + 1); + strncpy(priv->info.product_rev, id_data.product_rev, PRODUCT_REV_LENGTH + 1); + strncpy(priv->info.product_name, _generate_product_name(id_data.product_id), PRODUCT_NAME_LENGTH + 1); return 0; } @@ -1169,6 +1169,57 @@ static int _register_key(void *device, unsigned char *key) return ret; } +/** SCSI command handling of REPORT SUPPORTED OPERATION CODES + */ +static int _cdb_rsoc(void* device, unsigned char *buf, uint32_t len) +{ + int ret = -EDEV_UNKNOWN; + int ret_ep = DEVICE_GOOD; + struct sg_data *priv = (struct sg_data*)device; + + sg_io_hdr_t req; + unsigned char cdb[CDB12_LEN]; + unsigned char sense[MAXSENSE]; + int timeout = 60; + char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "RSOC"; + char *msg = NULL; + + /* Zero out the CDB and the result buffer */ + ret = init_sg_io_header(&req); + if (ret < 0) + return ret; + + memset(cdb, 0, sizeof(cdb)); + memset(buf, 0, len); + memset(sense, 0, sizeof(sense)); + + /* Build CDB */ + cdb[0] = MAINTENANCE_IN; + cdb[1] = 0x0C; /* REPORT SUPPORTED OPERATION CODES */ + cdb[2] = 0x80; /* Fetch all commands with RCTD */ + ltfs_u32tobe(cdb + 6, len); /* allocation length */ + + /* Build request */ + req.dxfer_direction = SCSI_FROM_TARGET_TO_INITIATOR; + req.cmd_len = sizeof(cdb); + req.mx_sb_len = sizeof(sense); + req.dxfer_len = len; + req.dxferp = buf; + req.cmdp = cdb; + req.sbp = sense; + req.timeout = SGConversion(timeout); + req.usr_ptr = (void *)cmd_desc; + + ret = sg_issue_cdb_command(&priv->dev, &req, &msg); + if (ret < 0){ + ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); + if (ret_ep < 0) + ret = ret_ep; + } + + return ret; +} + /* Global functions */ int sg_open(const char *devname, void **handle) { @@ -1180,6 +1231,9 @@ int sg_open(const char *devname, void **handle) struct sg_data *priv; + unsigned char *rsoc_buf = NULL; + uint32_t rsoc_len = RSOC_BUF_SIZE; + CHECK_ARG_NULL(devname, -LTFS_NULL_ARG); CHECK_ARG_NULL(handle, -LTFS_NULL_ARG); @@ -1323,11 +1377,46 @@ int sg_open(const char *devname, void **handle) increment_openfactor(priv->info.host, priv->info.channel); - /* Setup vendor specific parameters */ + /* Setup error table sense to ltfs error code */ init_error_table(priv->vendor, &standard_table, &vendor_table); - init_timeout(priv->vendor, &priv->timeouts, priv->drive_type); - if (!priv->timeouts) - ibm_tape_init_timeout(&priv->timeouts, priv->drive_type); + + /* Setup device specific timeout value */ + rsoc_buf = calloc(1, RSOC_BUF_SIZE); + if (rsoc_buf) { + ret = _cdb_rsoc(&priv->dev, rsoc_buf, RSOC_BUF_SIZE); + rsoc_len = ltfs_betou32(rsoc_buf); + if (!ret && rsoc_len < RSOC_BUF_SIZE) { + ltfsmsg(LTFS_INFO, 30294I, "RSOC"); + ret = init_timeout_rsoc(&priv->timeouts, rsoc_buf, rsoc_len); + if (!priv->timeouts) + ibm_tape_init_timeout(&priv->timeouts, priv->drive_type); + } + + if (ret < 0) { + /* + * The drive doesn't support RSOC, buffer overrun or parse error + * try to initialize the timeout table from drive vendor and drive type + */ + ltfsmsg(LTFS_INFO, 30294I, "vendor and device"); + ret = init_timeout(priv->vendor, &priv->timeouts, priv->drive_type); + if (!priv->timeouts) { + ltfsmsg(LTFS_INFO, 30294I, "device"); + ibm_tape_init_timeout(&priv->timeouts, priv->drive_type); + } + } + free(rsoc_buf); + } else { + /* + * Memory allocation failure, try to initialize the timeout table + * from drive vendor and drive type + */ + ltfsmsg(LTFS_INFO, 30294I, "vendor and device"); + init_timeout(priv->vendor, &priv->timeouts, priv->drive_type); + if (!priv->timeouts) { + ltfsmsg(LTFS_INFO, 30294I, "device"); + ibm_tape_init_timeout(&priv->timeouts, priv->drive_type); + } + } /* Issue TURs to clear POR sense */ _clear_por(priv); @@ -4258,12 +4347,12 @@ int sg_get_device_list(struct tc_drive_info *buf, int count) } if (found < count && buf) { - snprintf(buf[found].name, TAPE_DEVNAME_LEN_MAX + 1, "%s", devname); - snprintf(buf[found].vendor, TAPE_VENDOR_NAME_LEN_MAX + 1, "%s", identifier.vendor_id); - snprintf(buf[found].model, TAPE_MODEL_NAME_LEN_MAX + 1, "%s", identifier.product_id); - snprintf(buf[found].serial_number, TAPE_SERIAL_LEN_MAX + 1, "%s", identifier.unit_serial); - snprintf(buf[found].product_rev, PRODUCT_REV_LENGTH + 1, "%s", identifier.product_rev); - snprintf(buf[found].product_name, PRODUCT_NAME_LENGTH + 1, "%s", _generate_product_name(identifier.product_id)); + strncpy(buf[found].name, devname, TAPE_DEVNAME_LEN_MAX + 1); + strncpy(buf[found].vendor, identifier.vendor_id, TAPE_VENDOR_NAME_LEN_MAX + 1); + strncpy(buf[found].model, identifier.product_id, TAPE_MODEL_NAME_LEN_MAX + 1); + strncpy(buf[found].serial_number, identifier.unit_serial, TAPE_SERIAL_LEN_MAX + 1); + strncpy(buf[found].product_rev, identifier.product_rev, PRODUCT_REV_LENGTH + 1); + strncpy(buf[found].product_name, _generate_product_name(identifier.product_id), PRODUCT_NAME_LENGTH + 1); if (! ioctl(dev.fd, SG_GET_SCSI_ID, &scsi_id)) { buf[found].host = scsi_id.host_no; diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index 43854e97..619fd444 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -96,7 +96,10 @@ struct iokit_global_data global_data; #define LOG_PAGE_PARAM_OFFSET (4) #define IOKIT_MAX_BLOCK_SIZE (1 * MB) + +#ifndef MIN #define MIN(a, b) ((a) < (b) ? (a) : (b)) +#endif /* Forward references (For keep function order to struct tape_ops) */ int iokit_readpos(void *device, struct tc_position *pos); @@ -140,7 +143,6 @@ static inline int _parse_logPage(const unsigned char *logdata, i += param_len + LOG_PAGE_PARAM_OFFSET; } -out: return ret; } @@ -731,6 +733,51 @@ static int _register_key(void *device, unsigned char *key) return ret; } +/** SCSI command handling of REPORT SUPPORTED OPERATION CODES + */ +static int _cdb_rsoc(void* device, unsigned char *buf, uint32_t len) +{ + int ret = -EDEV_UNKNOWN; + struct iokit_data *priv = (struct iokit_data*)device; + + struct iokit_scsi_request req; + unsigned char cdb[CDB12_LEN]; + unsigned char sense[MAXSENSE]; + int timeout = 60; + char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "RSOC"; + char *msg = NULL; + + /* Zero out the CDB and the result buffer */ + memset(cdb, 0, sizeof(cdb)); + memset(&req, 0, sizeof(struct iokit_scsi_request)); + memset(buf, 0, len); + memset(sense, 0, sizeof(sense)); + + /* Build CDB */ + cdb[0] = MAINTENANCE_IN; + cdb[1] = 0x0C; /* REPORT SUPPORTED OPERATION CODES */ + cdb[2] = 0x80; /* Fetch all commands with RCTD */ + ltfs_u32tobe(cdb + 6, len); /* allocation length */ + + /* Build request */ + req.dxfer_direction = SCSI_FROM_TARGET_TO_INITIATOR; + req.cmd_len = sizeof(cdb); + req.mx_sb_len = sizeof(SCSI_Sense_Data); + req.dxfer_len = len; + req.dxferp = buf; + req.cmdp = cdb; + memset(&req.sense_buffer, 0, req.mx_sb_len); + req.timeout = IOKitConversion(timeout); + req.desc = cmd_desc; + + ret = iokit_issue_cdb_command(&priv->dev, &req, &msg); + if (ret < 0){ + _process_errors(device, ret, msg, cmd_desc, true); + } + + return ret; +} + /* Global functions */ int iokit_open(const char *devname, void **handle) { @@ -742,6 +789,9 @@ int iokit_open(const char *devname, void **handle) struct iokit_data *priv; scsi_device_identifier id_data; + unsigned char *rsoc_buf = NULL; + uint32_t rsoc_len = RSOC_BUF_SIZE; + ltfsmsg(LTFS_INFO, 30810I, devname); priv = calloc(1, sizeof(struct iokit_data)); @@ -850,9 +900,44 @@ int iokit_open(const char *devname, void **handle) priv->info.target = 0; priv->info.lun = -1; - /* Setup vendor specific parameters */ + /* Setup error table sense to ltfs error code */ init_error_table(priv->vendor, &standard_table, &vendor_table); - init_timeout(priv->vendor, &priv->timeouts, priv->drive_type); + + /* Setup device specific timeout value */ + rsoc_buf = calloc(1, RSOC_BUF_SIZE); + if (rsoc_buf) { + ret = _cdb_rsoc(&priv->dev, rsoc_buf, RSOC_BUF_SIZE); + rsoc_len = ltfs_betou32(rsoc_buf); + if (!ret && rsoc_len < RSOC_BUF_SIZE) { + ltfsmsg(LTFS_INFO, 30872I, "RSOC"); + ret = init_timeout_rsoc(&priv->timeouts, rsoc_buf, rsoc_len); + } + + if (ret < 0) { + /* + * The drive doesn't support RSOC, buffer overrun or parse error + * try to initialize the timeout table from drive vendor and drive type + */ + ltfsmsg(LTFS_INFO, 30872I, "vendor and device"); + ret = init_timeout(priv->vendor, &priv->timeouts, priv->drive_type); + if (!priv->timeouts) { + ltfsmsg(LTFS_INFO, 30872I, "device"); + ibm_tape_init_timeout(&priv->timeouts, priv->drive_type); + } + } + free(rsoc_buf); + } else { + /* + * Memory allocation failure, try to initialize the timeout table + * from drive vendor and drive type + */ + ltfsmsg(LTFS_INFO, 30872I, "vendor and device"); + init_timeout(priv->vendor, &priv->timeouts, priv->drive_type); + if (!priv->timeouts) { + ltfsmsg(LTFS_INFO, 30872I, "device"); + ibm_tape_init_timeout(&priv->timeouts, priv->drive_type); + } + } /* Register reservation key */ ibm_tape_genkey(priv->key); @@ -880,7 +965,6 @@ int iokit_open(const char *devname, void **handle) int iokit_reopen(const char *devname, void *device) { - char *end; int drive_type = DRIVE_UNSUPPORTED; int ret = -EDEV_UNKNOWN; @@ -931,7 +1015,7 @@ int iokit_reopen(const char *devname, void *device) } else priv->drive_type = drive_type; } else { - ltfsmsg(LTFS_INFO, 30813I, id_data.product_id); + ltfsmsg(LTFS_INFO, 30813I, id_data.vendor_id, id_data.product_id); iokit_release_exclusive_access(&priv->dev); ret = -EDEV_DEVICE_UNSUPPORTABLE; /* Unsupported device */ } @@ -1202,10 +1286,10 @@ static int _cdb_read(void *device, char *buf, size_t size, boolean_t sili) * In this case, LTFS will trust SCSI sense. */ if (diff_len < 0) { - ltfsmsg(LTFS_INFO, 30820I, diff_len, size - diff_len); // "Detect overrun condition" + ltfsmsg(LTFS_INFO, 30820I, diff_len, (int)(size - diff_len)); // "Detect overrun condition" ret = -EDEV_OVERRUN; } else { - ltfsmsg(LTFS_DEBUG, 30821D, diff_len, size - diff_len); // "Detect underrun condition" + ltfsmsg(LTFS_DEBUG, 30821D, diff_len, (int)(size - diff_len)); // "Detect underrun condition" length = size - diff_len; ret = DEVICE_GOOD; } @@ -1215,10 +1299,10 @@ static int _cdb_read(void *device, char *buf, size_t size, boolean_t sili) #endif } else { if (diff_len < 0) { - ltfsmsg(LTFS_INFO, 30820I, diff_len, size - diff_len); // "Detect overrun condition" + ltfsmsg(LTFS_INFO, 30820I, diff_len, (int)(size - diff_len)); // "Detect overrun condition" ret = -EDEV_OVERRUN; } else { - ltfsmsg(LTFS_DEBUG, 30821D, diff_len, size - diff_len); // "Detect underrun condition" + ltfsmsg(LTFS_DEBUG, 30821D, diff_len, (int)(size - diff_len)); // "Detect underrun condition" length = size - diff_len; ret = DEVICE_GOOD; } @@ -1301,7 +1385,8 @@ int iokit_read(void *device, char *buf, size_t size, return ret; } goto read_retry; - } else if ( !(pos->block) && unusual_size && (ret == size || ret == -EDEV_FILEMARK_DETECTED) ) { + } else if ( !(pos->block) && unusual_size && + (ret == (int32_t)size || ret == -EDEV_FILEMARK_DETECTED) ) { /* * Try to read again without sili bit, because some I/F doesn't support SILION read correctly * like @@ -1552,7 +1637,7 @@ int iokit_rewind(void *device, struct tc_position *pos) char *msg = NULL; ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_REWIND)); - ltfsmsg(LTFS_DEBUG, 30997D, "rewind", 0, 0, priv->drive_serial); + ltfsmsg(LTFS_DEBUG, 30997D, "rewind", (unsigned long long)0, (unsigned long long)0, priv->drive_serial); // Zero out the CDB and the result buffer memset(cdb, 0, sizeof(cdb)); @@ -1615,7 +1700,7 @@ int iokit_locate(void *device, struct tc_position dest, struct tc_position *pos) bool pc = false; ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOCATE)); - ltfsmsg(LTFS_DEBUG, 30997D, "locate", dest.partition, dest.block, priv->drive_serial); + ltfsmsg(LTFS_DEBUG, 30997D, "locate", (unsigned long long)dest.partition, dest.block, priv->drive_serial); if (pos->partition != dest.partition) { if (priv->clear_by_pc) { @@ -1868,7 +1953,7 @@ int iokit_erase(void *device, struct tc_position *pos, bool long_erase) if (IS_ENTERPRISE(priv->drive_type)) { get_current_timespec(&ts_now); - ltfsmsg(LTFS_INFO, 30829I, (ts_now.tv_sec - ts_start.tv_sec)/60); + ltfsmsg(LTFS_INFO, 30829I, (int)((ts_now.tv_sec - ts_start.tv_sec)/60)); } else { progress = ((uint32_t) sense_buf[16] & 0xFF) << 8; progress += ((uint32_t) sense_buf[17] & 0xFF); @@ -2295,7 +2380,7 @@ int iokit_remaining_capacity(void *device, struct tc_remaining_cap *cap) offset = (int)buf[0] + 1; length = (int)buf[offset] + 1; - if (offset + length <= param_size) { + if ((uint32_t)(offset + length) <= param_size) { cap->max_p1 = ltfs_betou32(&buf[offset + PARTITIOIN_REC_HEADER_LEN]); } @@ -2310,7 +2395,7 @@ int iokit_remaining_capacity(void *device, struct tc_remaining_cap *cap) offset = (int)buf[0] + 1; length = (int)buf[offset] + 1; - if (offset + length <= param_size) { + if ((uint32_t)(offset + length) <= param_size) { cap->remaining_p1 = ltfs_betou32(&buf[offset + PARTITIOIN_REC_HEADER_LEN]); } @@ -2539,7 +2624,7 @@ int iokit_reserve(void *device) int count = 0; ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_RESERVEUNIT)); - ltfsmsg(LTFS_DEBUG, 30392D, "reserve (PRO)", priv->drive_serial); + ltfsmsg(LTFS_DEBUG, 30992D, "reserve (PRO)", priv->drive_serial); start: ret = _cdb_pro(device, PRO_ACT_RESERVE, PRO_TYPE_EXCLUSIVE, @@ -2609,7 +2694,7 @@ int iokit_release(void *device) #else /* Use persistent reserve */ ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_RELEASEUNIT)); - ltfsmsg(LTFS_DEBUG, 30392D, "release (PRO)", priv->drive_serial); + ltfsmsg(LTFS_DEBUG, 30992D, "release (PRO)", priv->drive_serial); ret = _cdb_pro(device, PRO_ACT_RELEASE, PRO_TYPE_EXCLUSIVE, priv->key, NULL); @@ -2766,7 +2851,7 @@ int iokit_read_attribute(void *device, const tape_partition_t part, char *msg = NULL; ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_READATTR)); - ltfsmsg(LTFS_DEBUG3, 30997D, "readattr", (unsigned long)part, id, priv->drive_serial); + ltfsmsg(LTFS_DEBUG3, 30997D, "readattr", (unsigned long long)part, (unsigned long long)id, priv->drive_serial); /* Prepare the buffer to transfer */ uint32_t len = size + 4; @@ -2842,7 +2927,7 @@ int iokit_allow_overwrite(void *device, const struct tc_position pos) char *msg = NULL; ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_ALLOWOVERW)); - ltfsmsg(LTFS_DEBUG, 30997D, "allow overwrite", pos.partition, pos.block, priv->drive_serial); + ltfsmsg(LTFS_DEBUG, 30997D, "allow overwrite", (unsigned long long)pos.partition, pos.block, priv->drive_serial); // Zero out the CDB and the result buffer memset(cdb, 0, sizeof(cdb)); @@ -3521,7 +3606,7 @@ int iokit_get_device_list(struct tc_drive_info *buf, int count) int i, ret; int found = 0; int32_t devs = iokit_get_ssc_device_count(); - uint32_t drive_type; + int drive_type; scsi_device_identifier identifier; struct iokit_device *iokit_device; // Pointer to device status structure @@ -3540,7 +3625,7 @@ int iokit_get_device_list(struct tc_drive_info *buf, int count) continue; } drive_type = iokit_get_drive_identifier(iokit_device, &identifier); - if (drive_type != -1) { + if (!drive_type) { if (found < count && buf) { snprintf(buf[i].name, TAPE_DEVNAME_LEN_MAX, "%d", i); snprintf(buf[i].vendor, TAPE_VENDOR_NAME_LEN_MAX, "%s", identifier.vendor_id); @@ -3970,7 +4055,6 @@ int iokit_get_keyalias(void *device, unsigned char **keyalias) free: free(buffer); -out: ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_GETKEYALIAS)); return ret; } @@ -4156,7 +4240,7 @@ int iokit_get_block_in_buffer(void *device, uint32_t *block) *block = (buf[5] << 16) + (buf[6] << 8) + (int)buf[7]; ltfsmsg(LTFS_DEBUG, 30998D, "blocks-in-buffer", - (unsigned long long) *block, 0, 0, priv->drive_serial); + (unsigned long long) *block, (unsigned long long)0, (unsigned long long)0, priv->drive_serial); } else { _process_errors(device, ret, msg, cmd_desc, true); } diff --git a/src/tape_drivers/tape_drivers.h b/src/tape_drivers/tape_drivers.h index 1d087710..19ddf895 100644 --- a/src/tape_drivers/tape_drivers.h +++ b/src/tape_drivers/tape_drivers.h @@ -69,6 +69,11 @@ #define REDPOS_LONG_LEN (32) #define REDPOS_EXT_LEN (32) +#define RSOC_BUF_SIZE (4 * KB) +#define RSOC_ENT_SIZE (20) +#define RSOC_HEADER_SIZE (4) +#define RSOC_RECOM_TO_OFFSET (16) + #ifndef MAXSENSE #define MAXSENSE (255) #endif @@ -111,9 +116,9 @@ static inline int _sense2errorcode(uint32_t sense, struct error_table *table, ch if ( (sense & 0xFFFF00) == 0x044000 ) sense = 0x044000; - else if ( (sense & 0xFFF000) == 0x048000 ) /* 04/8xxx in TS3100/TS3200 */ + else if ( (sense & 0xFFFF00) == 0x048000 ) /* 04/8xxx in TS3100/TS3200 */ sense = 0x048000; - else if ( (sense & 0xFFF000) == 0x0B4100 ) /* 0B/41xx in TS2900 */ + else if ( (sense & 0xFFFF00) == 0x0B4100 ) /* 0B/41xx in TS2900 */ sense = 0x0B4100; if ( (sense & 0x00FF00) >= 0x008000 || (sense & 0x0000FF) >= 0x000080) diff --git a/src/tape_drivers/vendor_compat.c b/src/tape_drivers/vendor_compat.c index 482479db..7a0acbb1 100644 --- a/src/tape_drivers/vendor_compat.c +++ b/src/tape_drivers/vendor_compat.c @@ -428,6 +428,39 @@ int init_timeout(int vendor, struct timeout_tape **table, int type) return ret; } +int init_timeout_rsoc(struct timeout_tape **table, unsigned char *buf, uint32_t len) +{ + int entries = len / RSOC_ENT_SIZE; + int modulo = len % RSOC_ENT_SIZE; + int i = 0, offset = 0; + int op_code, timeout; + struct timeout_tape *entry, *out = NULL; + + if (modulo) { + /* Descriptor length is invalid */ + return -EDEV_INVALID_ARG; + } + + HASH_CLEAR(hh, *table); + + for (i = 0; i < entries; i++) { + offset = RSOC_HEADER_SIZE + (RSOC_ENT_SIZE * i); + op_code = (int)(*((unsigned char *)(buf + offset))); + timeout = (int)(ltfs_betou32(buf + offset + RSOC_RECOM_TO_OFFSET)); + + out = NULL; + HASH_FIND_INT(*table, &op_code, out); + if (!out) { + entry = malloc(sizeof(struct timeout_tape)); + entry->op_code = op_code; + entry->timeout = timeout; + HASH_ADD_INT(*table, op_code, entry); + } + } + + return 0; +} + void destroy_timeout(struct timeout_tape** table) { struct timeout_tape *entry, *tmp; diff --git a/src/tape_drivers/vendor_compat.h b/src/tape_drivers/vendor_compat.h index 08c1e245..e6eec141 100644 --- a/src/tape_drivers/vendor_compat.h +++ b/src/tape_drivers/vendor_compat.h @@ -76,6 +76,7 @@ void init_error_table(int vendor, struct error_table **vendor_table); int init_timeout(int vendor, struct timeout_tape **table, int type); +int init_timeout_rsoc(struct timeout_tape **table, unsigned char *buf, uint32_t len); void destroy_timeout(struct timeout_tape **table); int get_timeout(struct timeout_tape *table, int op_code); From 1fbde528e7aa0ce2a253fbd8511dd91064c66cd6 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 10 Mar 2021 21:30:59 +0900 Subject: [PATCH 021/121] Introduce generic logselect issue I/F via VEA (#258) - Introduce generic logselect issue I/F via EA by ltfs.vendor.IBM.logPage.XX.YY - XX shall be a hex value like A0 as page code. YY XX shall be a hex value like 60 as subpage code - Once LTFS receives a correct VEA, XX and YY can be converted by strroul() with base=16, LTFS always issues logsense to the drive and return the specified page by binary --- src/libltfs/ltfs.c | 30 ++++- src/libltfs/ltfs.h | 3 +- src/libltfs/tape.c | 12 ++ src/libltfs/tape.h | 2 + src/libltfs/tape_ops.h | 10 +- src/libltfs/xattr.c | 24 ++++ src/tape_drivers/freebsd/cam/cam_tc.c | 79 ++++++----- src/tape_drivers/generic/file/filedebug_tc.c | 2 +- src/tape_drivers/generic/itdtimg/itdtimg_tc.c | 2 +- .../linux/lin_tape/lin_tape_ibmtape.c | 38 +++--- src/tape_drivers/linux/sg/sg_tape.c | 70 ++++++---- .../netbsd/scsipi-ibmtape/scsipi_ibmtape.c | 58 ++++---- src/tape_drivers/osx/iokit/iokit_tape.c | 126 ++++++++++-------- src/tape_drivers/tape_drivers.h | 2 + 14 files changed, 287 insertions(+), 171 deletions(-) diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 8c3a7855..f5d1dcdd 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -4039,9 +4039,37 @@ int ltfs_release_medium(struct ltfs_volume *vol) return 0; } +/** + * Capture log page. + * + * At buffer shortage, this function returns the length of the page. The page is truncated + * by size. + * + * @param vol LTFS volume + * @param page log page to capture + * @param subpage subpage to capture + * @param buf buffer of the contents of logpage + * @param size buffer size + * @return page length on success or a negative value on error + */ +int ltfs_logpage(const uint8_t page, const uint8_t subpage, unsigned char *buf, + const size_t size, struct ltfs_volume *vol) +{ + int ret = -EDEV_UNKNOWN; + + if (vol->device) { + tape_device_lock(vol->device); + ret = tape_logsense(vol->device, page, subpage, buf, size); + tape_device_unlock(vol->device); + } + + return ret; +} + /** * Wait the drive goes to ready state - * @param device handle to tape device + * + * @param vol LTFS volume * @return 0 on success or a negative value on error */ int ltfs_wait_device_ready(struct ltfs_volume *vol) diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index 1d411dd2..8444ff73 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -672,7 +672,6 @@ void ltfs_unset_index_dirty(bool update_version, struct ltfs_index *idx); int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol); int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, struct ltfs_volume *vol); - char ltfs_dp_id(struct ltfs_volume *vol); char ltfs_ip_id(struct ltfs_volume *vol); const char *ltfs_get_volume_uuid(struct ltfs_volume *vol); @@ -688,6 +687,8 @@ int ltfs_traverse_index_no_eod(struct ltfs_volume *vol, char partition, unsigned int ltfs_check_eod_status(struct ltfs_volume *vol); int ltfs_recover_eod(struct ltfs_volume *vol); int ltfs_release_medium(struct ltfs_volume *vol); +int ltfs_logpage(const uint8_t page, const uint8_t subpage, unsigned char *buf, + const size_t size, struct ltfs_volume *vol); int ltfs_wait_device_ready(struct ltfs_volume *vol); void ltfs_recover_eod_simple(struct ltfs_volume *vol); diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index 8fcab456..caf33222 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -2611,6 +2611,18 @@ int tape_is_cartridge_loadable(struct device_data *dev) return ret; } +int tape_logsense(struct device_data *dev, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size) +{ + int ret = -EDEV_UNKNOWN; + + CHECK_ARG_NULL(dev, -LTFS_NULL_ARG); + + ret = dev->backend->logsense(dev->backend_data, page, subpage, buf, size); + + return ret; +} + /** * Wait the drive goes to ready state * @param device handle to tape device diff --git a/src/libltfs/tape.h b/src/libltfs/tape.h index c3131578..0a4a9454 100644 --- a/src/libltfs/tape.h +++ b/src/libltfs/tape.h @@ -203,6 +203,8 @@ int tape_enable_append_only_mode(struct device_data *dev, bool enable); int tape_get_append_only_mode_setting(struct device_data *dev, bool *enabled); int tape_is_cartridge_loadable(struct device_data *dev); +int tape_logsense(struct device_data *dev, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size); int tape_wait_device_ready(struct device_data *dev, void * const kmi_handle); int tape_set_key(struct device_data *dev, const unsigned char *keyalias, const unsigned char *key); int tape_clear_key(struct device_data *device, void * const kmi_handle); diff --git a/src/libltfs/tape_ops.h b/src/libltfs/tape_ops.h index 80b7e801..1acbbbdf 100644 --- a/src/libltfs/tape_ops.h +++ b/src/libltfs/tape_ops.h @@ -581,6 +581,7 @@ struct tape_ops { /** * Get capacity data from a device. + * * @param device Device handle returned by the backend's open(). * @param cap On success, the backend must fill this structure with the total and remaining * capacity values of the two partitions on the medium, in units of 1048576 bytes. @@ -590,16 +591,17 @@ struct tape_ops { /** * Send a SCSI Log Sense command to a device. - * libltfs does not currently use this function, but it may be useful internally (for example, - * ibmtape uses it from its remaining_capacity() function). + * * @param device Device handle returned by the backend's open(). * @param page Log page to query. + * @param subpage Specify the sub page of the log page to query. * @param buf On success, the backend must fill this buffer with the log page's value. * @param size Buffer size. - * @return 0 on success or a negative value on error. Backends for which Log Sense is + * @return Page length on success or a negative value on error. Backends for which Log Sense is * meaningless should return -1. */ - int (*logsense)(void *device, const uint8_t page, unsigned char *buf, const size_t size); + int (*logsense)(void *device, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size); /** * Send a SCSI Mode Sense(10) command to a device. diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index a4cacc1c..17dc6ad2 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -805,6 +805,7 @@ bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_volume *v || ! strcmp(name, "ltfs.vendor.IBM.cartridgeMountNode") || ! strcmp(name, "ltfs.vendor.IBM.logLevel") || ! strcmp(name, "ltfs.vendor.IBM.syslogLevel") + || ! strcmp(name, "ltfs.vendor.IBM.logPage") || ! strncmp(name, "ltfs.vendor", strlen("ltfs.vendor"))) return true; } @@ -1126,6 +1127,29 @@ int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, const char val = NULL; ret = -LTFS_NO_MEMORY; } + } else if ( (!strncmp(name, "ltfs.vendor.IBM.logPage.", strlen("ltfs.vendor.IBM.logPage."))) && + (strlen(name) == strlen("ltfs.vendor.IBM.logPage.XX.XX")) ) { + char page_str[3] = {0x00, 0x00, 0x00}; + char subpage_str[3] = {0x00, 0x00, 0x00}; + + uint8_t page = 0xFF; + uint8_t subpage = 0xFF; + + char *endptr = NULL; + + page_str[0] = name[24]; + page_str[1] = name[25]; + subpage_str[0] = name[27]; + subpage_str[1] = name[28]; + + page = (uint8_t)(strtoul(page_str, &endptr, 16)); + if (*endptr) return -LTFS_NO_XATTR; + + subpage = (uint8_t)(strtoul(subpage_str, &endptr, 16)); + if (*endptr) return -LTFS_NO_XATTR; + + ret = ltfs_logpage(page, subpage, (unsigned char *)buf, buf_size, vol); + } else if (! strncmp(name, "ltfs.vendor", strlen("ltfs.vendor"))) { if (! strncmp(name + strlen("ltfs.vendor."), LTFS_VENDOR_NAME, strlen(LTFS_VENDOR_NAME))) { ret = _xattr_get_vendorunique_xattr(&val, name, vol); diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index e3333b15..22db821d 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -1686,8 +1686,8 @@ int camtape_format(void *device, TC_FORMAT_TYPE format, const char *vol_name, co */ #define MAX_UINT16 (0x0000FFFF) -int camtape_logsense_page(struct camtape_data *softc, const uint8_t page, const uint8_t subpage, - unsigned char *buf, const size_t size) +int camtape_logsense(struct camtape_data *softc, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size) { int rc = DEVICE_GOOD; char *msg = NULL; @@ -1695,8 +1695,16 @@ int camtape_logsense_page(struct camtape_data *softc, const uint8_t page, const union ccb *ccb = NULL; int timeout; - ltfsmsg(LTFS_DEBUG3, 31397D, "logsense", (unsigned long long)page, (unsigned long long)subpage, - softc->drive_serial); + unsigned int len = 0; + unsigned char *inner_buf = NULL; + + ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); + ltfsmsg(LTFS_DEBUG3, 31397D, "logsense", + (unsigned long long)page, (unsigned long long)subpage, softc->drive_serial); + + inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 1MB */ + if (!inner_buf) + return -LTFS_NO_MEMORY; ccb = cam_getccb(softc->cd); if (ccb == NULL) { @@ -1722,8 +1730,8 @@ int camtape_logsense_page(struct camtape_data *softc, const uint8_t page, const /*save_pages*/ 0, /*ppc*/ 0, /*paramptr*/ 0, - /*param_buf*/ buf, - /*param_len*/ size, + /*param_buf*/ inner_buf, + /*param_len*/ MAXLP_SIZE, /*sense_len*/ SSD_FULL_SIZE, /*timeout*/ timeout); /* @@ -1738,23 +1746,27 @@ int camtape_logsense_page(struct camtape_data *softc, const uint8_t page, const if (rc != DEVICE_GOOD) camtape_process_errors(softc, rc, msg, "logsense page", true); + else { + len = ((int)inner_buf[2] << 8) + (int)inner_buf[3] + 4; + + if (size > len) + memcpy(buf, inner_buf, len); + else + memcpy(buf, inner_buf, size); + + rc = len; + } bailout: + if (inner_buf != NULL) + free(inner_buf); + if (ccb != NULL) cam_freeccb(ccb); - return rc; -} - -int camtape_logsense(void *device, const uint8_t page, unsigned char *buf, const size_t size) -{ - struct camtape_data *softc = (struct camtape_data *)device; - int ret = 0; - - ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); - ret = camtape_logsense_page(softc, page, 0, buf, size); ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); - return ret; + + return rc; } #define PARTITIOIN_REC_HEADER_LEN (4) @@ -1775,8 +1787,8 @@ int camtape_remaining_capacity(void *device, struct tc_remaining_cap *cap) (softc->vendor == VENDOR_HP && IS_LTO(softc->drive_type) && (DRIVE_GEN(softc->drive_type) == 0x06))) { /* Issue LogPage 0x31 */ - rc = camtape_logsense(device, LOG_TAPECAPACITY, logdata, LOGSENSEPAGE); - if (rc) { + rc = camtape_logsense(device, LOG_TAPECAPACITY, (uint8_t)0, logdata, LOGSENSEPAGE); + if (rc < 0) { ltfsmsg(LTFS_INFO, 31257I, LOG_TAPECAPACITY, rc); ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_REMAINCAP)); return rc; @@ -1815,8 +1827,8 @@ int camtape_remaining_capacity(void *device, struct tc_remaining_cap *cap) } else { /* Issue LogPage 0x17 */ - rc = camtape_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); - if (rc) { + rc = camtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); + if (rc < 0) { ltfsmsg(LTFS_INFO, 31257I, LOG_VOLUMESTATS, rc); ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_REMAINCAP)); return rc; @@ -2467,8 +2479,8 @@ int camtape_get_cartridge_health(void *device, struct tc_cartridge_health *cart_ /* Issue LogPage 0x37 */ cart_health->tape_efficiency = UNSUPPORTED_CARTRIDGE_HEALTH; - rc = camtape_logsense(device, LOG_PERFORMANCE, logdata, LOGSENSEPAGE); - if (rc) + rc = camtape_logsense(device, LOG_PERFORMANCE, (uint8_t)0, logdata, LOGSENSEPAGE); + if (rc < 0) ltfsmsg(LTFS_INFO, 31261I, LOG_PERFORMANCE, rc, "get cart health"); else { for(i = 0; i < (int)((sizeof(perfstats)/sizeof(perfstats[0]))); i++) { /* BEAM: loop doesn't iterate - Use loop for future enhancement. */ @@ -2518,8 +2530,9 @@ int camtape_get_cartridge_health(void *device, struct tc_cartridge_health *cart_ cart_health->read_mbytes = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_begin = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_middle = UNSUPPORTED_CARTRIDGE_HEALTH; - rc = camtape_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); - if (rc) + + rc = camtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); + if (rc < 0) ltfsmsg(LTFS_INFO, 31261I, LOG_VOLUMESTATS, rc, "get cart health"); else { for(i = 0; i < (int)((sizeof(volstats)/sizeof(volstats[0]))); i++) { @@ -2615,10 +2628,11 @@ int camtape_get_tape_alert(void *device, uint64_t *tape_alert) ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_GETTAPEALT)); /* Issue LogPage 0x2E */ ta = 0; - rc = camtape_logsense(device, LOG_TAPE_ALERT, logdata, LOGSENSEPAGE); - if (rc) + rc = camtape_logsense(device, LOG_TAPE_ALERT, (uint8_t)0, logdata, LOGSENSEPAGE); + if (rc < 0) ltfsmsg(LTFS_INFO, 31261I, LOG_TAPE_ALERT, rc, "get tape alert"); else { + rc = 0; for(i = 1; i <= 64; i++) { if (parse_logPage(logdata, (uint16_t) i, ¶m_size, buf, 16) || param_size != sizeof(uint8_t)) { @@ -3065,8 +3079,8 @@ int camtape_get_eod_status(void *device, int part) ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_GETEODSTAT)); /* Issue LogPage 0x17 */ - rc = camtape_logsense(device, LOG_VOL_STATISTICS, logdata, LOGSENSEPAGE); - if (rc) { + rc = camtape_logsense(device, LOG_VOL_STATISTICS, (uint8_t)0, logdata, LOGSENSEPAGE); + if (rc < 0) { ltfsmsg(LTFS_WARN, 31264W, LOG_VOL_STATISTICS, rc); ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_GETEODSTAT)); return EOD_UNKNOWN; @@ -3134,11 +3148,12 @@ int camtape_get_xattr(void *device, const char *name, char **buf) get_current_timespec(&now); if ( softc->fetch_sec_acq_loss_w == 0 || ((softc->fetch_sec_acq_loss_w + 60 < now.tv_sec) && softc->dirty_acq_loss_w)) { - rc = camtape_logsense_page(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, - logdata, LOGSENSEPAGE); - if (rc) + rc = camtape_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, + logdata, LOGSENSEPAGE); + if (rc < 0) ltfsmsg(LTFS_INFO, 31261I, LOG_PERFORMANCE, rc, "get xattr"); else { + rc = 0; if (parse_logPage(logdata, PERF_ACTIVE_CQ_LOSS_W, ¶m_size, logbuf, 16)) { ltfsmsg(LTFS_INFO, 31262I, LOG_PERFORMANCE, "get xattr"); rc = -LTFS_NO_XATTR; diff --git a/src/tape_drivers/generic/file/filedebug_tc.c b/src/tape_drivers/generic/file/filedebug_tc.c index f08418a5..10191c3c 100644 --- a/src/tape_drivers/generic/file/filedebug_tc.c +++ b/src/tape_drivers/generic/file/filedebug_tc.c @@ -1906,7 +1906,7 @@ int filedebug_set_xattr(void *device, const char *name, const char *buf, size_t return ret; } -int filedebug_logsense(void *device, const uint8_t page, unsigned char *buf, const size_t size) +int filedebug_logsense(void *device, const uint8_t page, const uint8_t subpage, unsigned char *buf, const size_t size) { ltfsmsg(LTFS_ERR, 10007E, __FUNCTION__); return -EDEV_UNSUPPORTED_FUNCTION; diff --git a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c index dad017d7..7d6d106b 100644 --- a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c +++ b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c @@ -857,7 +857,7 @@ int itdtimage_set_xattr(void *device, const char *name, const char *buf, size_t return -LTFS_NO_XATTR; } -int itdtimage_logsense(void *device, const uint8_t page, unsigned char *buf, const size_t size) +int itdtimage_logsense(void *device, const uint8_t page, const uint8_t subpage, unsigned char *buf, const size_t size) { ltfsmsg(LTFS_ERR, 10007E, __FUNCTION__); return -EDEV_UNSUPPORTED_FUNCTION; diff --git a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c index 61e673c8..4239f729 100644 --- a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c +++ b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c @@ -2208,13 +2208,14 @@ int lin_tape_ibmtape_format(void *device, TC_FORMAT_TYPE format, const char *vol */ #define MAX_UINT16 (0x0000FFFF) -int lin_tape_ibmtape_logsense_page(void *device, const uint8_t page, const uint8_t subpage, - unsigned char *buf, const size_t size) +int lin_tape_ibmtape_logsense(void *device, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size) { int rc; char *msg; struct log_sense10_page log_page; + ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); ltfsmsg(LTFS_DEBUG3, 30597D, "logsense", (unsigned long long)page, (unsigned long long)subpage, ((struct lin_tape_ibmtape *) device)->drive_serial); @@ -2228,23 +2229,13 @@ int lin_tape_ibmtape_logsense_page(void *device, const uint8_t page, const uint8 if (rc != DEVICE_GOOD) { lin_tape_ibmtape_process_errors(device, rc, msg, "logsense page", true); - } - else { + } else { memcpy(buf, log_page.data, size); } - return rc; -} - -int lin_tape_ibmtape_logsense(void *device, const uint8_t page, unsigned char *buf, const size_t size) -{ - int ret = 0; - struct lin_tape_ibmtape *priv = (struct lin_tape_ibmtape *) device; - - ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); - ret = lin_tape_ibmtape_logsense_page(device, page, 0, buf, size); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); - return ret; + + return logpage.len; } /** @@ -2269,7 +2260,7 @@ int lin_tape_ibmtape_remaining_capacity(void *device, struct tc_remaining_cap *c ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_REMAINCAP)); if (IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) { /* Issue LogPage 0x31 */ - rc = lin_tape_ibmtape_logsense(device, LOG_TAPECAPACITY, logdata, LOGSENSEPAGE); + rc = lin_tape_ibmtape_logsense(device, LOG_TAPECAPACITY, (uint8_t)0, logdata, LOGSENSEPAGE); if (rc) { ltfsmsg(LTFS_INFO, 30457I, LOG_TAPECAPACITY, rc); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_REMAINCAP)); @@ -2309,7 +2300,7 @@ int lin_tape_ibmtape_remaining_capacity(void *device, struct tc_remaining_cap *c } else { /* Issue LogPage 0x17 */ - rc = lin_tape_ibmtape_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); + rc = lin_tape_ibmtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); if (rc) { ltfsmsg(LTFS_INFO, 30457I, LOG_VOLUMESTATS, rc); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_REMAINCAP)); @@ -2904,7 +2895,7 @@ int lin_tape_ibmtape_get_cartridge_health(void *device, struct tc_cartridge_heal /* Issue LogPage 0x37 */ cart_health->tape_efficiency = UNSUPPORTED_CARTRIDGE_HEALTH; - rc = lin_tape_ibmtape_logsense(device, LOG_PERFORMANCE, logdata, LOGSENSEPAGE); + rc = lin_tape_ibmtape_logsense(device, LOG_PERFORMANCE, (uint8_t)0, logdata, LOGSENSEPAGE); if (rc) ltfsmsg(LTFS_INFO, 30461I, LOG_PERFORMANCE, rc, "get cart health"); else { @@ -2955,7 +2946,8 @@ int lin_tape_ibmtape_get_cartridge_health(void *device, struct tc_cartridge_heal cart_health->read_mbytes = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_begin = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_middle = UNSUPPORTED_CARTRIDGE_HEALTH; - rc = lin_tape_ibmtape_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); + + rc = lin_tape_ibmtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); if (rc) ltfsmsg(LTFS_INFO, 30461I, LOG_VOLUMESTATS, rc, "get cart health"); else { @@ -3052,7 +3044,7 @@ int lin_tape_ibmtape_get_tape_alert(void *device, uint64_t *tape_alert) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_GETTAPEALT)); /* Issue LogPage 0x2E */ ta = 0; - rc = lin_tape_ibmtape_logsense(device, LOG_TAPE_ALERT, logdata, LOGSENSEPAGE); + rc = lin_tape_ibmtape_logsense(device, LOG_TAPE_ALERT, (uint8_t)0, logdata, LOGSENSEPAGE); if (rc) ltfsmsg(LTFS_INFO, 30461I, LOG_TAPE_ALERT, rc, "get tape alert"); else { @@ -3393,7 +3385,7 @@ int lin_tape_ibmtape_get_eod_status(void *device, int part) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_GETEODSTAT)); /* Issue LogPage 0x17 */ - rc = lin_tape_ibmtape_logsense(device, LOG_VOL_STATISTICS, logdata, LOGSENSEPAGE); + rc = lin_tape_ibmtape_logsense(device, LOG_VOL_STATISTICS, (uint8_t)0, logdata, LOGSENSEPAGE); if (rc) { ltfsmsg(LTFS_WARN, 30464W, LOG_VOL_STATISTICS, rc); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_GETEODSTAT)); @@ -3462,8 +3454,8 @@ int lin_tape_ibmtape_get_xattr(void *device, const char *name, char **buf) get_current_timespec(&now); if ( priv->fetch_sec_acq_loss_w == 0 || ((priv->fetch_sec_acq_loss_w + 60 < now.tv_sec) && priv->dirty_acq_loss_w)) { - rc = lin_tape_ibmtape_logsense_page(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, - logdata, LOGSENSEPAGE); + rc = lin_tape_ibmtape_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, + logdata, LOGSENSEPAGE); if (rc) ltfsmsg(LTFS_INFO, 30461I, LOG_PERFORMANCE, rc, "get xattr"); else { diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index 0f2f9de6..d5d5752d 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -107,7 +107,7 @@ struct sg_global_data global_data; int sg_readpos(void *device, struct tc_position *pos); int sg_locate(void *device, struct tc_position dest, struct tc_position *pos); int sg_space(void *device, size_t count, TC_SPACE_TYPE type, struct tc_position *pos); -int sg_logsense(void *device, const unsigned char page, unsigned char *buf, const size_t size); +int sg_logsense(void *device, const uint8_t page, const uint8_t subpage, unsigned char *buf, const size_t size); int sg_modesense(void *device, const unsigned char page, const TC_MP_PC_TYPE pc, const unsigned char subpage, unsigned char *buf, const size_t size); int sg_modeselect(void *device, unsigned char *buf, const size_t size); @@ -2972,7 +2972,7 @@ int sg_remaining_capacity(void *device, struct tc_remaining_cap *cap) (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))) { /* Use LogPage 0x31 */ - ret = sg_logsense(device, (uint8_t)LOG_TAPECAPACITY, (void *)buffer, LOGSENSEPAGE); + ret = sg_logsense(device, (uint8_t)LOG_TAPECAPACITY, (uint8_t)0, (void *)buffer, LOGSENSEPAGE); if(ret < 0) { ltfsmsg(LTFS_INFO, 30229I, LOG_VOLUMESTATS, ret); @@ -3027,7 +3027,7 @@ int sg_remaining_capacity(void *device, struct tc_remaining_cap *cap) ret = DEVICE_GOOD; } else { /* Use LogPage 0x17 */ - ret = sg_logsense(device, LOG_VOLUMESTATS, (void *)buffer, LOGSENSEPAGE); + ret = sg_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, (void *)buffer, LOGSENSEPAGE); if(ret < 0) { ltfsmsg(LTFS_INFO, 30229I, LOG_VOLUMESTATS, ret); @@ -3096,8 +3096,8 @@ int sg_remaining_capacity(void *device, struct tc_remaining_cap *cap) return ret; } -static int _cdb_logsense(void *device, const unsigned char page, const unsigned char subpage, - unsigned char *buf, const size_t size) +int sg_logsense(void *device, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size) { int ret = -EDEV_UNKNOWN; int ret_ep = DEVICE_GOOD; @@ -3110,12 +3110,23 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "LOGSENSE"; char *msg = NULL; + unsigned int len = 0; + unsigned char *inner_buf = NULL; + ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); + ltfsmsg(LTFS_DEBUG3, 30397D, "logsense", + (unsigned long long)page, (unsigned long long)subpage, priv->drive_serial); + + inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 1MB */ + if (!inner_buf) + return -LTFS_NO_MEMORY; /* Zero out the CDB and the result buffer */ ret = init_sg_io_header(&req); - if (ret < 0) + if (ret < 0) { + free(inner_buf); return ret; + } memset(cdb, 0, sizeof(cdb)); memset(sense, 0, sizeof(sense)); @@ -3124,18 +3135,20 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned cdb[0] = LOG_SENSE; cdb[2] = 0x40 | (page & 0x3F); /* Current value */ cdb[3] = subpage; - ltfs_u16tobe(cdb + 7, size); + ltfs_u16tobe(cdb + 7, MAXLP_SIZE); timeout = get_timeout(priv->timeouts, cdb[0]); - if (timeout < 0) + if (timeout < 0) { + free(inner_buf); return -EDEV_UNSUPPORETD_COMMAND; + } /* Build request */ req.dxfer_direction = SCSI_FROM_TARGET_TO_INITIATOR; req.cmd_len = sizeof(cdb); req.mx_sb_len = sizeof(sense); - req.dxfer_len = size; - req.dxferp = buf; + req.dxfer_len = MAXLP_SIZE; + req.dxferp = inner_buf; req.cmdp = cdb; req.sbp = sense; req.timeout = SGConversion(timeout); @@ -3146,19 +3159,19 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); if (ret_ep < 0) ret = ret_ep; - } - - ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); + } else { + len = ((int)inner_buf[2] << 8) + (int)inner_buf[3] + 4; - return ret; -} + if (size > len) + memcpy(buf, inner_buf, len); + else + memcpy(buf, inner_buf, size); -int sg_logsense(void *device, const unsigned char page, unsigned char *buf, const size_t size) -{ - int ret = -EDEV_UNKNOWN; + ret = len; + } - ltfsmsg(LTFS_DEBUG3, 30393D, "logsense", page, ""); - ret = _cdb_logsense(device, page, 0x00, buf, size); + free (inner_buf); + ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); return ret; } @@ -3763,8 +3776,8 @@ int sg_get_cartridge_health(void *device, struct tc_cartridge_health *cart_healt /* Issue LogPage 0x37 */ cart_health->tape_efficiency = UNSUPPORTED_CARTRIDGE_HEALTH; - ret = sg_logsense(device, LOG_PERFORMANCE, logdata, LOGSENSEPAGE); - if (ret) + ret = sg_logsense(device, LOG_PERFORMANCE, (uint8_t)0, logdata, LOGSENSEPAGE); + if (ret < 0) ltfsmsg(LTFS_INFO, 30234I, LOG_PERFORMANCE, ret, "get cart health"); else { for(i = 0; i < (int)((sizeof(perfstats)/sizeof(perfstats[0]))); i++) { @@ -3818,7 +3831,7 @@ int sg_get_cartridge_health(void *device, struct tc_cartridge_health *cart_healt cart_health->read_mbytes = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_begin = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_middle = UNSUPPORTED_CARTRIDGE_HEALTH; - ret = sg_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); + ret = sg_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); if (ret < 0) ltfsmsg(LTFS_INFO, 30234I, LOG_VOLUMESTATS, ret, "get cart health"); else { @@ -3914,10 +3927,11 @@ int sg_get_tape_alert(void *device, uint64_t *tape_alert) /* Issue LogPage 0x2E */ ta = 0; - ret = sg_logsense(device, LOG_TAPE_ALERT, logdata, LOGSENSEPAGE); + ret = sg_logsense(device, LOG_TAPE_ALERT, (uint8_t)0, logdata, LOGSENSEPAGE); if (ret < 0) ltfsmsg(LTFS_INFO, 30234I, LOG_TAPE_ALERT, ret, "get tape alert"); else { + ret = 0; for(i = 1; i <= 64; i++) { if (_parse_logPage(logdata, (uint16_t) i, ¶m_size, buf, 16) || param_size != sizeof(uint8_t)) { @@ -3970,11 +3984,11 @@ int sg_get_xattr(void *device, const char *name, char **buf) if (priv->fetch_sec_acq_loss_w == 0 || ((priv->fetch_sec_acq_loss_w + 60 < now.tv_sec) && priv->dirty_acq_loss_w)) { - ret = _cdb_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, logdata, LOGSENSEPAGE); - + ret = sg_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, logdata, LOGSENSEPAGE); if (ret < 0) { ltfsmsg(LTFS_INFO, 30234I, LOG_PERFORMANCE, ret, "get xattr"); } else { + ret = 0; if (_parse_logPage(logdata, PERF_ACTIVE_CQ_LOSS_W, ¶m_size, logbuf, 16)) { ltfsmsg(LTFS_INFO, 30235I, LOG_PERFORMANCE, "get xattr"); ret = -LTFS_NO_XATTR; @@ -4225,8 +4239,8 @@ int sg_get_eod_status(void *device, int part) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_GETEODSTAT)); /* Issue LogPage 0x17 */ - ret = sg_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); - if (ret) { + ret = sg_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); + if (ret < 0) { ltfsmsg(LTFS_WARN, 30237W, LOG_VOLUMESTATS, ret); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_GETEODSTAT)); return EOD_UNKNOWN; diff --git a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c index 8dc7399a..8d11eb99 100644 --- a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c +++ b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c @@ -101,7 +101,8 @@ struct scsipi_ibmtape_global_data global_data; int scsipi_ibmtape_readpos(void *device, struct tc_position *pos); int scsipi_ibmtape_locate(void *device, struct tc_position dest, struct tc_position *pos); int scsipi_ibmtape_space(void *device, size_t count, TC_SPACE_TYPE type, struct tc_position *pos); -int scsipi_ibmtape_logsense(void *device, const unsigned char page, unsigned char *buf, const size_t size); +int scsipi_ibmtape_logsense(void *device, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size); int scsipi_ibmtape_modesense(void *device, const unsigned char page, const TC_MP_PC_TYPE pc, const unsigned char subpage, unsigned char *buf, const size_t size); int scsipi_ibmtape_modeselect(void *device, unsigned char *buf, const size_t size); @@ -2576,7 +2577,7 @@ int scsipi_ibmtape_remaining_capacity(void *device, struct tc_remaining_cap *cap if (IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) { /* Use LogPage 0x31 */ - ret = scsipi_ibmtape_logsense(device, (uint8_t)LOG_TAPECAPACITY, (void *)buffer, LOGSENSEPAGE); + ret = scsipi_ibmtape_logsense(device, (uint8_t)LOG_TAPECAPACITY, (uint8_t)0, (void *)buffer, LOGSENSEPAGE); if(ret < 0) { ltfsmsg(LTFS_INFO, 30229I, LOG_VOLUMESTATS, ret); @@ -2631,7 +2632,7 @@ int scsipi_ibmtape_remaining_capacity(void *device, struct tc_remaining_cap *cap ret = DEVICE_GOOD; } else { /* Use LogPage 0x17 */ - ret = scsipi_ibmtape_logsense(device, LOG_VOLUMESTATS, (void *)buffer, LOGSENSEPAGE); + ret = scsipi_ibmtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, (void *)buffer, LOGSENSEPAGE); if(ret < 0) { ltfsmsg(LTFS_INFO, 30229I, LOG_VOLUMESTATS, ret); @@ -2700,8 +2701,8 @@ int scsipi_ibmtape_remaining_capacity(void *device, struct tc_remaining_cap *cap return ret; } -static int _cdb_logsense(void *device, const unsigned char page, const unsigned char subpage, - unsigned char *buf, const size_t size) +int scsipi_ibmtape_logsense(void *device, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size) { int ret = -EDEV_UNKNOWN; int ret_ep = DEVICE_GOOD; @@ -2713,8 +2714,16 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "LOGSENSE"; char *msg = NULL; + unsigned int len = 0; + unsigned char *inner_buf = NULL; + + ltfsmsg(LTFS_DEBUG3, 30397D, "logsense", page, subpage, priv->drive_serial); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); + inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 1MB */ + if (!inner_buf) + return -LTFS_NO_MEMORY; + /* Zero out the CDB and the result buffer */ ret = init_scsireq(&req); if (ret < 0) @@ -2726,7 +2735,7 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned cdb[0] = LOG_SENSE; cdb[2] = 0x40 | (page & 0x3F); /* Current value */ cdb[3] = subpage; - ltfs_u16tobe(cdb + 7, size); + ltfs_u16tobe(cdb + 7, MAXLP_SIZE); timeout = ibm_tape_get_timeout(priv->timeouts, cdb[0]); if (timeout < 0) @@ -2745,19 +2754,19 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); if (ret_ep < 0) ret = ret_ep; - } - - ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); + } else { + len = ((int)inner_buf[2] << 8) + (int)inner_buf[3] + 4; - return ret; -} + if (size > len) + memcpy(buf, inner_buf, len); + else + memcpy(buf, inner_buf, size); -int scsipi_ibmtape_logsense(void *device, const unsigned char page, unsigned char *buf, const size_t size) -{ - int ret = -EDEV_UNKNOWN; + ret = len; + } - ltfsmsg(LTFS_DEBUG3, 30393D, "logsense", page, ""); - ret = _cdb_logsense(device, page, 0x00, buf, size); + free (inner_buf); + ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); return ret; } @@ -3325,8 +3334,8 @@ int scsipi_ibmtape_get_cartridge_health(void *device, struct tc_cartridge_health /* Issue LogPage 0x37 */ cart_health->tape_efficiency = UNSUPPORTED_CARTRIDGE_HEALTH; - ret = scsipi_ibmtape_logsense(device, LOG_PERFORMANCE, logdata, LOGSENSEPAGE); - if (ret) + ret = scsipi_ibmtape_logsense(device, LOG_PERFORMANCE, (uint8_t)0, logdata, LOGSENSEPAGE); + if (ret < 0) ltfsmsg(LTFS_INFO, 30234I, LOG_PERFORMANCE, ret, "get cart health"); else { for(i = 0; i < (int)((sizeof(perfstats)/sizeof(perfstats[0]))); i++) { @@ -3380,7 +3389,8 @@ int scsipi_ibmtape_get_cartridge_health(void *device, struct tc_cartridge_health cart_health->read_mbytes = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_begin = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_middle = UNSUPPORTED_CARTRIDGE_HEALTH; - ret = scsipi_ibmtape_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); + + ret = scsipi_ibmtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); if (ret < 0) ltfsmsg(LTFS_INFO, 30234I, LOG_VOLUMESTATS, ret, "get cart health"); else { @@ -3476,10 +3486,11 @@ int scsipi_ibmtape_get_tape_alert(void *device, uint64_t *tape_alert) /* Issue LogPage 0x2E */ ta = 0; - ret = scsipi_ibmtape_logsense(device, LOG_TAPE_ALERT, logdata, LOGSENSEPAGE); + ret = scsipi_ibmtape_logsense(device, LOG_TAPE_ALERT, (uint8_t)0, logdata, LOGSENSEPAGE); if (ret < 0) ltfsmsg(LTFS_INFO, 30234I, LOG_TAPE_ALERT, ret, "get tape alert"); else { + ret = 0; for(i = 1; i <= 64; i++) { if (_parse_logPage(logdata, (uint16_t) i, ¶m_size, buf, 16) || param_size != sizeof(uint8_t)) { @@ -3532,11 +3543,12 @@ int scsipi_ibmtape_get_xattr(void *device, const char *name, char **buf) if (priv->fetch_sec_acq_loss_w == 0 || ((priv->fetch_sec_acq_loss_w + 60 < now.tv_sec) && priv->dirty_acq_loss_w)) { - ret = _cdb_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, logdata, LOGSENSEPAGE); + ret = scsipi_ibmtape_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, logdata, LOGSENSEPAGE); if (ret < 0) { ltfsmsg(LTFS_INFO, 30234I, LOG_PERFORMANCE, ret, "get xattr"); } else { + ret = 0; if (_parse_logPage(logdata, PERF_ACTIVE_CQ_LOSS_W, ¶m_size, logbuf, 16)) { ltfsmsg(LTFS_INFO, 30235I, LOG_PERFORMANCE, "get xattr"); ret = -LTFS_NO_XATTR; @@ -3782,8 +3794,8 @@ int scsipi_ibmtape_get_eod_status(void *device, int part) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_GETEODSTAT)); /* Issue LogPage 0x17 */ - ret = scsipi_ibmtape_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); - if (ret) { + ret = scsipi_ibmtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); + if (ret < 0) { ltfsmsg(LTFS_WARN, 30237W, LOG_VOLUMESTATS, ret); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_GETEODSTAT)); return EOD_UNKNOWN; diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index 619fd444..8417477b 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -104,7 +104,8 @@ struct iokit_global_data global_data; /* Forward references (For keep function order to struct tape_ops) */ int iokit_readpos(void *device, struct tc_position *pos); int iokit_locate(void *device, struct tc_position dest, struct tc_position *pos); -int iokit_logsense(void *device, const unsigned char page, unsigned char *buf, const size_t size); +int iokit_logsense(void *device, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size); int iokit_modesense(void *device, const unsigned char page, const TC_MP_PC_TYPE pc, const unsigned char subpage, unsigned char *buf, const size_t size); int iokit_modeselect(void *device, unsigned char *buf, const size_t size); @@ -457,7 +458,7 @@ static int _cdb_read_buffer(void *device, int id, unsigned char *buf, size_t off ltfsmsg(LTFS_DEBUG, 30993D, "read buffer", id, priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -509,7 +510,7 @@ static int _cdb_force_dump(struct iokit_data *priv) ltfsmsg(LTFS_DEBUG, 30993D, "force dump", 0, priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); memset(&buf, 0, sizeof(buf)); @@ -1095,7 +1096,7 @@ int iokit_inquiry_page(void *device, unsigned char page, struct tc_inq_page *inq ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_INQUIRYPAGE)); ltfsmsg(LTFS_DEBUG, 30993D, "inquiry", page, priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1174,7 +1175,7 @@ int iokit_test_unit_ready(void *device) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_TUR)); ltfsmsg(LTFS_DEBUG3, 30992D, "test unit ready", priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1238,7 +1239,7 @@ static int _cdb_read(void *device, char *buf, size_t size, boolean_t sili) char *msg = NULL; size_t length = -EDEV_UNKNOWN; - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1439,7 +1440,7 @@ static int _cdb_write(void *device, uint8_t *buf, size_t size, bool *ew, bool *p char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "WRITE"; char *msg = NULL; - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1559,7 +1560,7 @@ int iokit_writefm(void *device, size_t count, struct tc_position *pos, bool imme ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_WRITEFM)); ltfsmsg(LTFS_DEBUG, 30994D, "write file marks", count, priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1639,7 +1640,7 @@ int iokit_rewind(void *device, struct tc_position *pos) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_REWIND)); ltfsmsg(LTFS_DEBUG, 30997D, "rewind", (unsigned long long)0, (unsigned long long)0, priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1714,7 +1715,7 @@ int iokit_locate(void *device, struct tc_position dest, struct tc_position *pos) pc = true; } - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1777,7 +1778,7 @@ int iokit_space(void *device, size_t count, TC_SPACE_TYPE type, struct tc_positi ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_SPACE)); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1860,7 +1861,7 @@ static int _cdb_request_sense(void *device, unsigned char *buf, unsigned char si char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "REQUEST_SENSE"; char *msg = NULL; - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1910,7 +1911,7 @@ int iokit_erase(void *device, struct tc_position *pos, bool long_erase) get_current_timespec(&ts_start); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -1984,7 +1985,7 @@ static int _cdb_load_unload(void *device, bool load) char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "LOAD_UNLOAD"; char *msg = NULL; - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2135,7 +2136,7 @@ int iokit_readpos(void *device, struct tc_position *pos) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_READPOS)); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2214,7 +2215,7 @@ int iokit_setcap(void *device, uint16_t proportion) ret = iokit_modeselect(device, buf, sizeof(buf)); } else { - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2260,7 +2261,7 @@ int iokit_format(void *device, TC_FORMAT_TYPE format, const char *vol_name, cons ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_FORMAT)); ltfsmsg(LTFS_DEBUG, 30992D, "format", priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2317,7 +2318,7 @@ int iokit_remaining_capacity(void *device, struct tc_remaining_cap *cap) (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))) { /* Use LogPage 0x31 */ - ret = iokit_logsense(device, (uint8_t)LOG_TAPECAPACITY, (void *)buffer, LOGSENSEPAGE); + ret = iokit_logsense(device, (uint8_t)LOG_TAPECAPACITY, (uint8_t)0, (void *)buffer, LOGSENSEPAGE); if(ret < 0) { ltfsmsg(LTFS_INFO, 30832I, LOG_VOLUMESTATS, ret); @@ -2360,9 +2361,8 @@ int iokit_remaining_capacity(void *device, struct tc_remaining_cap *cap) ret = DEVICE_GOOD; } else { /* Use LogPage 0x17 */ - ret = iokit_logsense(device, LOG_VOLUMESTATS, (void *)buffer, LOGSENSEPAGE); - if(ret < 0) - { + ret = iokit_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, (void *)buffer, LOGSENSEPAGE); + if(ret < 0) { ltfsmsg(LTFS_INFO, 30832I, LOG_VOLUMESTATS, ret); goto out; } @@ -2418,8 +2418,8 @@ int iokit_remaining_capacity(void *device, struct tc_remaining_cap *cap) return ret; } -static int _cdb_logsense(void *device, const unsigned char page, const unsigned char subpage, - unsigned char *buf, const size_t size) +int iokit_logsense(void *device, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size) { int ret = -EDEV_UNKNOWN; struct iokit_data *priv = (struct iokit_data*)device; @@ -2430,9 +2430,18 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "LOGSENSE"; char *msg = NULL; + unsigned int len = 0; + unsigned char *inner_buf = NULL; + ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); + ltfsmsg(LTFS_DEBUG3, 30997D, "logsense", + (unsigned long long)page, (unsigned long long)subpage, priv->drive_serial); + + inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 1MB */ + if (!inner_buf) + return -LTFS_NO_MEMORY; - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2440,18 +2449,20 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned cdb[0] = LOG_SENSE; cdb[2] = 0x40 | (page & 0x3F); /* Current value */ cdb[3] = subpage; - ltfs_u16tobe(cdb + 7, size); + ltfs_u16tobe(cdb + 7, MAXLP_SIZE); timeout = get_timeout(priv->timeouts, cdb[0]); - if (timeout < 0) + if (timeout < 0) { + free(inner_buf); return -EDEV_UNSUPPORETD_COMMAND; + } /* Build request */ req.dxfer_direction = SCSI_FROM_TARGET_TO_INITIATOR; req.cmd_len = sizeof(cdb); req.mx_sb_len = sizeof(SCSI_Sense_Data); - req.dxfer_len = size; - req.dxferp = buf; + req.dxfer_len = MAXLP_SIZE; + req.dxferp = inner_buf; req.cmdp = cdb; memset(&req.sense_buffer, 0, req.mx_sb_len); req.timeout = IOKitConversion(timeout); @@ -2460,19 +2471,19 @@ static int _cdb_logsense(void *device, const unsigned char page, const unsigned ret = iokit_issue_cdb_command(&priv->dev, &req, &msg); if (ret < 0){ _process_errors(device, ret, msg, cmd_desc, true); - } - - ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); + } else { + len = ((int)inner_buf[2] << 8) + (int)inner_buf[3] + 4; - return ret; -} + if (size > len) + memcpy(buf, inner_buf, len); + else + memcpy(buf, inner_buf, size); -int iokit_logsense(void *device, const unsigned char page, unsigned char *buf, const size_t size) -{ - int ret = -EDEV_UNKNOWN; + ret = len; + } - ltfsmsg(LTFS_DEBUG3, 30993D, "logsense", page, ""); - ret = _cdb_logsense(device, page, 0x00, buf, size); + free (inner_buf); + ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); return ret; } @@ -2492,7 +2503,7 @@ int iokit_modesense(void *device, const unsigned char page, const TC_MP_PC_TYPE ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_MODESENSE)); ltfsmsg(LTFS_DEBUG3, 30993D, "modesense", page, priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2543,7 +2554,7 @@ int iokit_modeselect(void *device, unsigned char *buf, const size_t size) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_MODESELECT)); ltfsmsg(LTFS_DEBUG3, 30992D, "modeselect", priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2592,7 +2603,7 @@ int iokit_reserve(void *device) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_RESERVEUNIT)); ltfsmsg(LTFS_DEBUG, 30992D, "reserve unit (6)", priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2664,7 +2675,7 @@ int iokit_release(void *device) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_RELEASEUNIT)); ltfsmsg(LTFS_DEBUG, 30992D, "release unit (6)", priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2717,7 +2728,7 @@ static int _cdb_prevent_allow_medium_removal(void *device, bool prevent) char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "PREVENT/ALLOW_MEDIUM_REMOVAL"; char *msg = NULL; - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2799,7 +2810,7 @@ int iokit_write_attribute(void *device, const tape_partition_t part, ltfs_u32tobe(buffer, len); memcpy(buffer + 4, buf, size); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2861,7 +2872,7 @@ int iokit_read_attribute(void *device, const tape_partition_t part, return -EDEV_NO_MEMORY; } - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -2929,7 +2940,7 @@ int iokit_allow_overwrite(void *device, const struct tc_position pos) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_ALLOWOVERW)); ltfsmsg(LTFS_DEBUG, 30997D, "allow overwrite", (unsigned long long)pos.partition, pos.block, priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -3094,8 +3105,8 @@ int iokit_get_cartridge_health(void *device, struct tc_cartridge_health *cart_he /* Issue LogPage 0x37 */ cart_health->tape_efficiency = UNSUPPORTED_CARTRIDGE_HEALTH; - ret = iokit_logsense(device, LOG_PERFORMANCE, logdata, LOGSENSEPAGE); - if (ret) + ret = iokit_logsense(device, LOG_PERFORMANCE, (uint8_t)0, logdata, LOGSENSEPAGE); + if (ret < 0) ltfsmsg(LTFS_INFO, 30837I, LOG_PERFORMANCE, ret, "get cart health"); else { for(i = 0; i < (int)((sizeof(perfstats)/sizeof(perfstats[0]))); i++) { @@ -3149,7 +3160,7 @@ int iokit_get_cartridge_health(void *device, struct tc_cartridge_health *cart_he cart_health->read_mbytes = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_begin = UNSUPPORTED_CARTRIDGE_HEALTH; cart_health->passes_middle = UNSUPPORTED_CARTRIDGE_HEALTH; - ret = iokit_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); + ret = iokit_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); if (ret < 0) ltfsmsg(LTFS_INFO, 30837I, LOG_VOLUMESTATS, ret, "get cart health"); else { @@ -3245,10 +3256,11 @@ int iokit_get_tape_alert(void *device, uint64_t *tape_alert) /* Issue LogPage 0x2E */ ta = 0; - ret = iokit_logsense(device, LOG_TAPE_ALERT, logdata, LOGSENSEPAGE); + ret = iokit_logsense(device, LOG_TAPE_ALERT, (uint8_t)0, logdata, LOGSENSEPAGE); if (ret < 0) ltfsmsg(LTFS_INFO, 30837I, LOG_TAPE_ALERT, ret, "get tape alert"); else { + ret = 0; for(i = 1; i <= 64; i++) { if (_parse_logPage(logdata, (uint16_t) i, ¶m_size, buf, 16) || param_size != sizeof(uint8_t)) { @@ -3301,11 +3313,11 @@ int iokit_get_xattr(void *device, const char *name, char **buf) if (priv->fetch_sec_acq_loss_w == 0 || ((priv->fetch_sec_acq_loss_w + 60 < now.tv_sec) && priv->dirty_acq_loss_w)) { - ret = _cdb_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, logdata, LOGSENSEPAGE); - + ret = iokit_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, logdata, LOGSENSEPAGE); if (ret < 0) { ltfsmsg(LTFS_INFO, 30837I, LOG_PERFORMANCE, ret, "get xattr"); } else { + ret = 0; if (_parse_logPage(logdata, PERF_ACTIVE_CQ_LOSS_W, ¶m_size, logbuf, 16)) { ltfsmsg(LTFS_INFO, 30838I, LOG_PERFORMANCE, "get xattr"); ret = -LTFS_NO_XATTR; @@ -3405,7 +3417,7 @@ static int _cdb_read_block_limits(void *device) { ltfsmsg(LTFS_DEBUG, 30992D, "read block limits", priv->drive_serial); - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -3533,8 +3545,8 @@ int iokit_get_eod_status(void *device, int part) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_GETEODSTAT)); /* Issue LogPage 0x17 */ - ret = iokit_logsense(device, LOG_VOLUMESTATS, logdata, LOGSENSEPAGE); - if (ret) { + ret = iokit_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); + if (ret < 0) { ltfsmsg(LTFS_WARN, 30840W, LOG_VOLUMESTATS, ret); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_GETEODSTAT)); return EOD_UNKNOWN; @@ -3698,7 +3710,7 @@ static int _cdb_spin(void *device, const uint16_t sps, unsigned char **buffer, s char *msg = NULL; size_t len = *size + 4; - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); @@ -3751,7 +3763,7 @@ int _cdb_spout(void *device, const uint16_t sps, char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "SPOUT"; char *msg = NULL; - // Zero out the CDB and the result buffer + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); memset(&req, 0, sizeof(struct iokit_scsi_request)); diff --git a/src/tape_drivers/tape_drivers.h b/src/tape_drivers/tape_drivers.h index 19ddf895..b75ccc24 100644 --- a/src/tape_drivers/tape_drivers.h +++ b/src/tape_drivers/tape_drivers.h @@ -78,6 +78,8 @@ #define MAXSENSE (255) #endif +#define MAXLP_SIZE (0xFFFF) + #define MASK_WITH_SENSE_KEY (0xFFFFFF) #define MASK_WITHOUT_SENSE_KEY (0x00FFFF) From 4be4c4078c10a98e0147c8ce6964f17d55baab0e Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Thu, 11 Mar 2021 10:11:26 +0900 Subject: [PATCH 022/121] Add another Quantum's LTO 6 into the list (#259) --- src/tape_drivers/quantum_tape.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tape_drivers/quantum_tape.c b/src/tape_drivers/quantum_tape.c index 504ee8d7..e475e4a2 100644 --- a/src/tape_drivers/quantum_tape.c +++ b/src/tape_drivers/quantum_tape.c @@ -68,6 +68,7 @@ struct supported_device *quantum_supported_drives[] = { TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH7", DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* QUANTUM Ultrium Gen 7 Half-High */ TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH8", DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* QUANTUM Ultrium Gen 8 Half-High */ TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 5", DRIVE_LTO5_HH, "[ULTRIUM-5]" ), /* Another QUANTUM Ultrium Gen 5 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 6", DRIVE_LTO5_HH, "[ULTRIUM-6]" ), /* Another QUANTUM Ultrium Gen 5 Half-High */ /* End of supported_devices */ NULL }; From 0bfb629a43ae29f4423b41c246ac76691b898c6c Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Sat, 13 Mar 2021 11:15:59 +0900 Subject: [PATCH 023/121] Fix plugin load error for the itdtimg backend (#260) --- src/tape_drivers/generic/itdtimg/itdtimg_tc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c index 7d6d106b..16625a18 100644 --- a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c +++ b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c @@ -1419,7 +1419,7 @@ int itdtimage_get_serialnumber(void *vstate, char **result) return DEVICE_GOOD; } -int filedebug_get_info(void *device, struct tc_drive_info *info) +int itdtimage_get_info(void *device, struct tc_drive_info *info) { /* * Return dummy data. @@ -1642,6 +1642,7 @@ struct tape_ops itdtimage_handler = { .is_mountable = itdtimage_is_mountable, .get_worm_status = itdtimage_get_worm_status, .get_serialnumber = itdtimage_get_serialnumber, + .get_info = itdtimage_get_info, .set_profiler = itdtimage_set_profiler, .get_block_in_buffer = itdtimage_get_block_in_buffer, .is_readonly = itdtimage_is_readonly, From 16a08be53dce3807aac6094e7295744f8f617d35 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Sat, 13 Mar 2021 11:56:05 +0900 Subject: [PATCH 024/121] Introduce generic MAM fields capture I/F (#261) - Introduce generic MAM fields capture I/F via ltfs.vendor.IBM.mediaMAM.XX - XX shall be one of 00, 01 , IP or DP - The new introduced VEA returns the MAM value by binary. You need to parse it by yourself --- src/libltfs/ltfs.c | 26 +++++++++++ src/libltfs/ltfs.h | 2 + src/libltfs/tape.c | 44 ++++++++++++++++++- src/libltfs/tape.h | 2 + src/libltfs/tape_ops.h | 2 + src/libltfs/xattr.c | 24 ++++++++++ src/tape_drivers/freebsd/cam/cam_tc.c | 7 ++- .../linux/lin_tape/lin_tape_ibmtape.c | 14 +++++- src/tape_drivers/linux/sg/sg_tape.c | 18 ++++++-- .../netbsd/scsipi-ibmtape/scsipi_ibmtape.c | 16 +++++-- src/tape_drivers/osx/iokit/iokit_tape.c | 16 +++++-- src/tape_drivers/tape_drivers.h | 1 + 12 files changed, 157 insertions(+), 15 deletions(-) diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index f5d1dcdd..f19da62f 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -4066,6 +4066,32 @@ int ltfs_logpage(const uint8_t page, const uint8_t subpage, unsigned char *buf, return ret; } +/** + * Capture MAM. + * + * At buffer shortage, this function returns the length of the page. The page is truncated + * by size. + * + * @param vol LTFS volume + * @param part partition to capture + * @param buf buffer of the contents of logpage + * @param size buffer size + * @return MAM length on success or a negative value on error + */ +int ltfs_mam(const tape_partition_t part, unsigned char *buf, + const size_t size, struct ltfs_volume *vol) +{ + int ret = -EDEV_UNKNOWN; + + if (vol->device) { + tape_device_lock(vol->device); + ret = tape_read_attr(vol->device, part, buf, size); + tape_device_unlock(vol->device); + } + + return ret; +} + /** * Wait the drive goes to ready state * diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index 8444ff73..f0e8bb9c 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -689,6 +689,8 @@ int ltfs_recover_eod(struct ltfs_volume *vol); int ltfs_release_medium(struct ltfs_volume *vol); int ltfs_logpage(const uint8_t page, const uint8_t subpage, unsigned char *buf, const size_t size, struct ltfs_volume *vol); +int ltfs_mam(const tape_partition_t part, unsigned char *buf, + const size_t size, struct ltfs_volume *vol); int ltfs_wait_device_ready(struct ltfs_volume *vol); void ltfs_recover_eod_simple(struct ltfs_volume *vol); diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index caf33222..c2017385 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -485,7 +485,7 @@ int tape_load_tape(struct device_data *dev, void * const kmi_handle, bool force) } /** - * Unroll operations made during tape_load_tape() + * Unroll operations made during tape_load_tape(). * @param dev device to unload * @return 0 on success or a negative value on error */ @@ -3472,7 +3472,47 @@ int read_tape_attribute(struct ltfs_volume *vol, char **val, const char *name) } /** - * Evaluate the tape can be mountable + * Generic I/F to get MAM. + * @param dev a pointer to the tape device + * @param part partition to get MAM + * @param buf pointer of the buffer + * @param size length of the buffer + * @return attr length on success or a negative value on error. + */ +int tape_read_attr(struct device_data *dev, const tape_partition_t part, + unsigned char *buf, const size_t size) +{ + int ret; + unsigned char *inner_buf = NULL; + unsigned int len = 0; + + CHECK_ARG_NULL(dev, -LTFS_NULL_ARG); + CHECK_ARG_NULL(dev->backend, -LTFS_NULL_ARG); + + inner_buf = calloc(1, MAXMAM_SIZE); /* Assume max length of MAM is 0xFFFF */ + if (!inner_buf) + return -LTFS_NO_MEMORY; + + ret = dev->backend->read_attribute(dev->backend_data, part, + 0, + inner_buf, + MAXMAM_SIZE); + if (!ret) { + len = ((int)inner_buf[2] << 8) + (int)inner_buf[3] + 4; + + if (size > len) + memcpy(buf, inner_buf, len); + else + memcpy(buf, inner_buf, size); + + ret = len; + } + + return ret; +} + +/** + * Evaluate the tape can be mountable. * @param device a pointer to the tape device * @param barcode barcode of the tape (NULL is accepted if cart_type and density is available) * @param cart_type cartridge type in CM diff --git a/src/libltfs/tape.h b/src/libltfs/tape.h index 0a4a9454..c202a92b 100644 --- a/src/libltfs/tape.h +++ b/src/libltfs/tape.h @@ -222,6 +222,8 @@ int tape_get_attribute_from_cm(struct device_data *dev, struct tape_attr *t_attr void tape_load_all_attribute_from_cm(struct device_data *dev, struct tape_attr *t_attr); int update_tape_attribute (struct ltfs_volume *vol, const char *new_value, int type, int size); int read_tape_attribute (struct ltfs_volume *vol, char **val, const char *name); +int tape_read_attr(struct device_data *dev, const tape_partition_t part, + unsigned char *buf, const size_t size); int tape_is_mountable(struct device_data *dev, char *barcode, unsigned char cart_type, unsigned char density); int tape_is_reformattable(struct device_data *dev, unsigned char cart_type, unsigned char density); diff --git a/src/libltfs/tape_ops.h b/src/libltfs/tape_ops.h index 1acbbbdf..b64b7ebb 100644 --- a/src/libltfs/tape_ops.h +++ b/src/libltfs/tape_ops.h @@ -666,6 +666,8 @@ struct tape_ops { * Read a MAM parameter from a device. * For performance reasons, it is recommended that all backends implement MAM parameter support. * However, this support is technically optional. + * Normally the buffer doesn't include header of contents. But it includes only when the size is + * MAXMAM_SIZE. * @param device Device handle returned by the backend's open(). * @param part Partition to read the parameter from. * @param id Attribute ID to read. libltfs uses TC_MAM_PAGE_VCR and TC_MAM_PAGE_COHERENCY. diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index 17dc6ad2..76c711b9 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -806,6 +806,7 @@ bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_volume *v || ! strcmp(name, "ltfs.vendor.IBM.logLevel") || ! strcmp(name, "ltfs.vendor.IBM.syslogLevel") || ! strcmp(name, "ltfs.vendor.IBM.logPage") + || ! strcmp(name, "ltfs.vendor.IBM.mediaMAM") || ! strncmp(name, "ltfs.vendor", strlen("ltfs.vendor"))) return true; } @@ -1150,6 +1151,29 @@ int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, const char ret = ltfs_logpage(page, subpage, (unsigned char *)buf, buf_size, vol); + } else if ( (!strncmp(name, "ltfs.vendor.IBM.mediaMAM.", strlen("ltfs.vendor.IBM.mediaMAM."))) && + (strlen(name) == strlen("ltfs.vendor.IBM.mediaMAM.XX")) ) { + char part_str[3] = {0x00, 0x00, 0x00}; + tape_partition_t part = 0; + + char *endptr = NULL; + + part_str[0] = name[25]; + part_str[1] = name[26]; + + if (!strncmp(part_str, "IP", sizeof(part_str))) { + part = ltfs_part_id2num(vol->label->partid_ip, vol); + } else if (!strncmp(part_str, "DP", sizeof(part_str))) { + part = ltfs_part_id2num(vol->label->partid_dp, vol);; + } else { + part = (uint8_t)(strtoul(part_str, &endptr, 16)); + if (*endptr) return -LTFS_NO_XATTR; + } + + if (part > 1) return -LTFS_NO_XATTR; + + ret = ltfs_mam(part, (unsigned char *)buf, buf_size, vol); + } else if (! strncmp(name, "ltfs.vendor", strlen("ltfs.vendor"))) { if (! strncmp(name + strlen("ltfs.vendor."), LTFS_VENDOR_NAME, strlen(LTFS_VENDOR_NAME))) { ret = _xattr_get_vendorunique_xattr(&val, name, vol); diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 22db821d..168db00a 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -1702,7 +1702,7 @@ int camtape_logsense(struct camtape_data *softc, const uint8_t page, const uint8 ltfsmsg(LTFS_DEBUG3, 31397D, "logsense", (unsigned long long)page, (unsigned long long)subpage, softc->drive_serial); - inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 1MB */ + inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 0xFFFF */ if (!inner_buf) return -LTFS_NO_MEMORY; @@ -2077,6 +2077,10 @@ int camtape_read_attribute(void *device, const tape_partition_t part, const uint size_t attr_size; union ccb *ccb = NULL; + /* TODO: Need to return data with header when size is MAXMAM_SIZE */ + if (size == MAXMAM_SIZE) + return -LTFS_NO_XATTR; + ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_READATTR)); ltfsmsg(LTFS_DEBUG3, 31397D, "readattr", (unsigned long long)part, (unsigned long long)id, softc->drive_serial); @@ -2144,6 +2148,7 @@ int camtape_read_attribute(void *device, const tape_partition_t part, const uint id != TC_MAM_APP_FORMAT_VERSION) ltfsmsg(LTFS_INFO, 31260I, rc); } else { + memcpy(buf, &attr_header[1], size); } diff --git a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c index 4239f729..7717eed3 100644 --- a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c +++ b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c @@ -2548,7 +2548,11 @@ int lin_tape_ibmtape_read_attribute(void *device, const tape_partition_t part, c memset(sense, 0, sizeof(sense)); /* Prepare Data Buffer */ - spt.buffer_length = size + 4; + if (size == MAXMAM_SIZE) + spt.buffer_length = MAXMAM_SIZE; + else + spt.buffer_length = size + 4; + spt.buffer = calloc(1, spt.buffer_length); if (spt.buffer == NULL) { ltfsmsg(LTFS_ERR, 10001E, "lin_tape_ibmtape_read_attribute: data buffer"); @@ -2593,7 +2597,13 @@ int lin_tape_ibmtape_read_attribute(void *device, const tape_partition_t part, c id != TC_MAM_APP_FORMAT_VERSION) ltfsmsg(LTFS_INFO, 30460I, rc); } else { - memcpy(buf, (spt.buffer + 4), size); + if (size == MAXMAM_SIZE) { + /* Include header if request size is MAXMAM_SIZE */ + memcpy(buf, spt.buffer, size); + } else { + memcpy(buf, (spt.buffer + 4), size); + } + free(spt.buffer); } diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index d5d5752d..bdab4a52 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -3117,7 +3117,7 @@ int sg_logsense(void *device, const uint8_t page, const uint8_t subpage, ltfsmsg(LTFS_DEBUG3, 30397D, "logsense", (unsigned long long)page, (unsigned long long)subpage, priv->drive_serial); - inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 1MB */ + inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 0xFFFF */ if (!inner_buf) return -LTFS_NO_MEMORY; @@ -3155,7 +3155,7 @@ int sg_logsense(void *device, const uint8_t page, const uint8_t subpage, req.usr_ptr = (void *)cmd_desc; ret = sg_issue_cdb_command(&priv->dev, &req, &msg); - if (ret < 0){ + if (ret < 0) { ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); if (ret_ep < 0) ret = ret_ep; @@ -3519,7 +3519,12 @@ int sg_read_attribute(void *device, const tape_partition_t part, ltfsmsg(LTFS_DEBUG3, 30397D, "readattr", (unsigned long long)part, (unsigned long long)id, priv->drive_serial); /* Prepare the buffer to transfer */ - uint32_t len = size + 4; + uint32_t len = 0; + if (size == MAXMAM_SIZE) + len = MAXMAM_SIZE; + else + len = size + 4; + unsigned char *buffer = calloc(1, len); if (!buffer) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); @@ -3577,7 +3582,12 @@ int sg_read_attribute(void *device, const tape_partition_t part, id != TC_MAM_APP_FORMAT_VERSION) ltfsmsg(LTFS_INFO, 30233I, ret); } else { - memcpy(buf, buffer + 4, size); + if (size == MAXMAM_SIZE) { + /* Include header if request size is MAXMAM_SIZE */ + memcpy(buf, buffer, size); + } else { + memcpy(buf, buffer + 4, size); + } } free(buffer); diff --git a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c index 8d11eb99..18765d1c 100644 --- a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c +++ b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c @@ -2720,7 +2720,7 @@ int scsipi_ibmtape_logsense(void *device, const uint8_t page, const uint8_t subp ltfsmsg(LTFS_DEBUG3, 30397D, "logsense", page, subpage, priv->drive_serial); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); - inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 1MB */ + inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 0xFFFF */ if (!inner_buf) return -LTFS_NO_MEMORY; @@ -3090,7 +3090,12 @@ int scsipi_ibmtape_read_attribute(void *device, const tape_partition_t part, ltfsmsg(LTFS_DEBUG3, 30397D, "readattr", (unsigned long long)part, (unsigned long long)id, priv->drive_serial); /* Prepare the buffer to transfer */ - uint32_t len = size + 4; + uint32_t len = 0; + if (size == MAXMAM_SIZE) + len = MAXMAM_SIZE; + else + len = size + 4; + unsigned char *buffer = calloc(1, len); if (!buffer) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); @@ -3144,7 +3149,12 @@ int scsipi_ibmtape_read_attribute(void *device, const tape_partition_t part, id != TC_MAM_APP_FORMAT_VERSION) ltfsmsg(LTFS_INFO, 30233I, ret); } else { - memcpy(buf, buffer + 4, size); + if (size == MAXMAM_SIZE) { + /* Include header if request size is MAXMAM_SIZE */ + memcpy(buf, buffer, size); + } else { + memcpy(buf, buffer + 4, size); + } } free(buffer); diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index 8417477b..ef25be56 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -2437,7 +2437,7 @@ int iokit_logsense(void *device, const uint8_t page, const uint8_t subpage, ltfsmsg(LTFS_DEBUG3, 30997D, "logsense", (unsigned long long)page, (unsigned long long)subpage, priv->drive_serial); - inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 1MB */ + inner_buf = calloc(1, MAXLP_SIZE); /* Assume max length of LP is 0xFFFF */ if (!inner_buf) return -LTFS_NO_MEMORY; @@ -2865,7 +2865,12 @@ int iokit_read_attribute(void *device, const tape_partition_t part, ltfsmsg(LTFS_DEBUG3, 30997D, "readattr", (unsigned long long)part, (unsigned long long)id, priv->drive_serial); /* Prepare the buffer to transfer */ - uint32_t len = size + 4; + uint32_t len = 0; + if (size == MAXMAM_SIZE) + len = MAXMAM_SIZE; + else + len = size + 4; + unsigned char *buffer = calloc(1, len); if (!buffer) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); @@ -2917,7 +2922,12 @@ int iokit_read_attribute(void *device, const tape_partition_t part, id != TC_MAM_APP_FORMAT_VERSION) ltfsmsg(LTFS_INFO, 30836I, ret); } else { - memcpy(buf, buffer + 4, size); + if (size == MAXMAM_SIZE) { + /* Include header if request size is MAXMAM_SIZE */ + memcpy(buf, buffer, size); + } else { + memcpy(buf, buffer + 4, size); + } } free(buffer); diff --git a/src/tape_drivers/tape_drivers.h b/src/tape_drivers/tape_drivers.h index b75ccc24..a9d7bf5e 100644 --- a/src/tape_drivers/tape_drivers.h +++ b/src/tape_drivers/tape_drivers.h @@ -79,6 +79,7 @@ #endif #define MAXLP_SIZE (0xFFFF) +#define MAXMAM_SIZE (0xFFFF) #define MASK_WITH_SENSE_KEY (0xFFFFFF) #define MASK_WITHOUT_SENSE_KEY (0x00FFFF) From 53a940605970711697faa34d46751016b97311f4 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Sat, 27 Mar 2021 20:01:57 +0900 Subject: [PATCH 025/121] Fix the build break of the lin_tape backend (#263) --- messages/internal_error/root.txt | 1 + src/libltfs/arch/errormap.c | 1 + src/libltfs/ltfs_error.h | 1 + src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c | 3 ++- 4 files changed, 5 insertions(+), 1 deletion(-) diff --git a/messages/internal_error/root.txt b/messages/internal_error/root.txt index 4ca96eec..7fb6c8ae 100644 --- a/messages/internal_error/root.txt +++ b/messages/internal_error/root.txt @@ -252,6 +252,7 @@ root:table { I1199E:string{ "You need to move tape to a storage slot first before the operation." } I1200E:string{ "The operation needs to be started again." } I1201E:string{ "Locate returns write-perm error." } + I1202E:string{ "Failed to open a stats DB." } // Special error codes I9997E:string{ "Child process error (ltfsck/mkltfs): %s (%d)." } diff --git a/src/libltfs/arch/errormap.c b/src/libltfs/arch/errormap.c index f3ad3d52..4ee2fa3b 100644 --- a/src/libltfs/arch/errormap.c +++ b/src/libltfs/arch/errormap.c @@ -287,6 +287,7 @@ static struct error_map fuse_error_list[] = { { LTFS_NEED_MOVE, "I1199E", EINVAL}, { LTFS_NEED_START_OVER, "I1200E", EINVAL}, { LTFS_LOCATE_ERROR, "I1201E", EIO}, + { LTFS_STATS_DB_OPEN, "I1202E", EIO}, { EDEV_NO_SENSE, "D0000E", EIO}, { EDEV_OVERRUN, "D0002E", EIO}, diff --git a/src/libltfs/ltfs_error.h b/src/libltfs/ltfs_error.h index d7aa6034..7d158d9e 100644 --- a/src/libltfs/ltfs_error.h +++ b/src/libltfs/ltfs_error.h @@ -258,6 +258,7 @@ #define LTFS_NEED_MOVE 1199 /* Need to move tape to a storage slot first before the operation */ #define LTFS_NEED_START_OVER 1200 /* Need operation needs to be started again */ #define LTFS_LOCATE_ERROR 1201 /* Locate returns write-perm error */ +#define LTFS_STATS_DB_OPEN 1202 /* Failed to open a stats DB */ #define LTFS_ERR_MAX 19999 diff --git a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c index 7717eed3..8985d39d 100644 --- a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c +++ b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c @@ -2214,6 +2214,7 @@ int lin_tape_ibmtape_logsense(void *device, const uint8_t page, const uint8_t su int rc; char *msg; struct log_sense10_page log_page; + struct lin_tape_ibmtape *priv = (struct lin_tape_ibmtape *) device; ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_LOGSENSE)); ltfsmsg(LTFS_DEBUG3, 30597D, "logsense", (unsigned long long)page, (unsigned long long)subpage, @@ -2235,7 +2236,7 @@ int lin_tape_ibmtape_logsense(void *device, const uint8_t page, const uint8_t su ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_LOGSENSE)); - return logpage.len; + return log_page.len; } /** From 3071374a2b7b0faa4d3ee75973cf669412104663 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 9 Apr 2021 18:05:25 +0900 Subject: [PATCH 026/121] Fix the drive type of Quantum LTO6 (#267) --- src/tape_drivers/quantum_tape.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tape_drivers/quantum_tape.c b/src/tape_drivers/quantum_tape.c index e475e4a2..880afbff 100644 --- a/src/tape_drivers/quantum_tape.c +++ b/src/tape_drivers/quantum_tape.c @@ -68,7 +68,7 @@ struct supported_device *quantum_supported_drives[] = { TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH7", DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* QUANTUM Ultrium Gen 7 Half-High */ TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH8", DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* QUANTUM Ultrium Gen 8 Half-High */ TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 5", DRIVE_LTO5_HH, "[ULTRIUM-5]" ), /* Another QUANTUM Ultrium Gen 5 Half-High */ - TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 6", DRIVE_LTO5_HH, "[ULTRIUM-6]" ), /* Another QUANTUM Ultrium Gen 5 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 6", DRIVE_LTO6_HH, "[ULTRIUM-6]" ), /* Another QUANTUM Ultrium Gen 6 Half-High */ /* End of supported_devices */ NULL }; From 7a2b2571b5986f012d948cfef197825abfa76b46 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Thu, 6 May 2021 18:11:19 +0900 Subject: [PATCH 027/121] Remove AC_CONFIG_MACRO_DIR from configure.ac (#274) --- configure.ac | 5 ++--- src/libltfs/arch/arch_info.c | 6 +++--- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/configure.ac b/configure.ac index b61ac3cb..c2d66b28 100644 --- a/configure.ac +++ b/configure.ac @@ -3,7 +3,7 @@ ;; OO_Copyright_BEGIN ;; ;; -;; Copyright 2010, 2020 IBM Corp. All rights reserved. +;; Copyright 2010, 2021 IBM Corp. All rights reserved. ;; ;; Redistribution and use in source and binary forms, with or without ;; modification, are permitted provided that the following conditions @@ -40,7 +40,6 @@ AC_INIT([LTFS], [2.5.0.0 (Prelim)], IBM corporation.) AC_CONFIG_SRCDIR([src/main.c]) AC_CONFIG_AUX_DIR([build-aux]) AC_CONFIG_MACRO_DIRS([m4]) -AC_CONFIG_MACRO_DIR([m4]) AC_CANONICAL_HOST AC_CANONICAL_BUILD AC_CANONICAL_TARGET @@ -391,7 +390,7 @@ fi dnl dnl Check for headers, types, structures, compiler characteristics dnl -AC_CHECK_HEADERS([fcntl.h limits.h stddef.h stdint.h stdlib.h string.h sys/ioctl.h sys/mount.h sys/time.h termios.h unistd.h]) +AC_CHECK_HEADERS([fcntl.h limits.h stddef.h stdint.h stdlib.h string.h sys/ioctl.h sys/mount.h sys/time.h termios.h unistd.h sys/sysctl.h]) AC_HEADER_STDBOOL AC_TYPE_MODE_T AC_TYPE_OFF_T diff --git a/src/libltfs/arch/arch_info.c b/src/libltfs/arch/arch_info.c index 1ee677b5..7a1fe1dc 100644 --- a/src/libltfs/arch/arch_info.c +++ b/src/libltfs/arch/arch_info.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2021 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -48,7 +48,7 @@ */ #include "libltfs/ltfs.h" -#ifndef mingw_PLATFORM +#ifdef HAVE_SYS_SYSCTL_H #include #endif #include @@ -133,7 +133,7 @@ void show_runtime_system_info(void) return; } -#elif defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) +#elif defined(HAVE_SYS_SYSCTL_H) { int mib[2]; size_t len; From 52594c66ba1aedc05b94dd0b59d072cba3d09d1b Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Thu, 6 May 2021 18:15:49 +0900 Subject: [PATCH 028/121] Update the README on master (#275) --- README.md | 275 +++++++++++++++++++++--------------------------------- 1 file changed, 104 insertions(+), 171 deletions(-) diff --git a/README.md b/README.md index 123247ce..2cc2507b 100644 --- a/README.md +++ b/README.md @@ -7,83 +7,13 @@ This is the `master` branch of the LTFS project. At this time, this branch is used for version 2.5 development. So it wouldn't be stable a little. Please consider to follow the tree on `v2.4-stable` branch if you want to use stable codes. -# Linear Tape File System (LTFS) +# What is the Linear Tape File System (LTFS) -Linear Tape File System (LTFS) is a filesystem to mount a LTFS formatted tape in a tape drive. Once LTFS mounts a LTFS formatted tape as filesystem, user can access to the tape via filesystem API. +The Linear Tape File System (LTFS) is a filesystem to mount a LTFS formatted tape in a tape drive. Once LTFS mounts a LTFS formatted tape as filesystem, user can access to the tape via filesystem API. Objective of this project is being the reference implementation of the LTFS format Specifications in [SNIA](https://www.snia.org/tech_activities/standards/curr_standards/ltfs). -At this time, the target of this project to meet is the LTFS format specifications 2.4. (https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf). - -## LTFS Format Specifications - -LTFS Format Specification is specified data placement, shape of index and names of extended attributes for LTFS. This specification is defined in [SNIA](https://www.snia.org/tech_activities/standards/curr_standards/ltfs) first and then it is forwarded to [ISO](https://www.iso.org/home.html) as ISO/IEC 20919 from version 2.2. - -The table below show status of the LTFS format Specification - - | Version | Status of SNIA | Status of ISO | - |:-: |:-: |:-: | - | 2.2 | [Published the Technical Position](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | Published | - | 2.3.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.pdf) | - | - | 2.4 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | On going | - | 2.5 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) | Not started yet | - -## How to use the LTFS (Quick start) - -This section is for person who already have a machine the LTFS is installed. - -Instruction how to use the LTFS is on [Wiki](https://github.com/LinearTapeFileSystem/ltfs/wiki). Please take a look! - -## Getting Started from GitHub project - -These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. - -## Prerequisites - -- Linux - * automake 1.13.4 or later - * autoconf 2.69 or later - * libtool 2.4.2 or later - * fuse 2.6.0 or later - * uuid 1.36 or later (Linux) - * libxml-2.0 2.6.16 or later - * net-snmp 5.3 or later - * icu4c 4.8 or later - -- OSX (macOS) - - Following packages on homebrew - - * automake - * autoconf - * libtool - * osxfuse (brew cask install osxfuse) - * ossp-uuid - * libxml2 - * icu4c - * gnu-sed - -- FreeBSD: - * FreeBSD 10.2 or 11.0 or later (for sa(4) driver changes) - * automake - * autoconf - * libtool - * fusefs-libs - * net-snmp - * e2fsprogs-libuuid - * libxml2 - * icu - -- NetBSD: - * NetBSD 7.0 or higher (for FUSE support) - * automake - * autoconf - * libtool - * libfuse - * net-snmp - * libuuid - * libxml2 - * icu +At this time, the target of this project to meet is the [LTFS format specifications 2.5](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf). ## Supported Tape Drives @@ -106,133 +36,133 @@ These instructions will get you a copy of the project up and running on your loc | Quantum | LTO7 (Only Half Height) | T.B.D. | | Quantum | LTO8 (Only Half Height) | T.B.D. | -## Installing +## LTFS Format Specifications -### Build and install on Linux +LTFS Format Specification is specified data placement, shape of index and names of extended attributes for LTFS. This specification is defined in [SNIA](https://www.snia.org/tech_activities/standards/curr_standards/ltfs) first and then it is forwarded to [ISO](https://www.iso.org/home.html) as ISO/IEC 20919 from version 2.2. -``` -./autogen.sh -./configure -make -make install -``` +The table below show status of the LTFS format Specification -`./configure --help` shows various options for build and install. + | Version | Status of SNIA | Status of ISO | + |:-------:|:----------------------------------------------------------------------------------------------------------------------------------------:|:---------------:| + | 2.2 | [Published the Technical Position](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | Published | + | 2.3.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.PDF) | - | + | 2.4 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | Published | + | 2.5.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) | Not started yet | -In some systems, you might need `sudo ldconfig -v` after `make install` to load the shared libraries correctly. +# How to use the LTFS (Quick start) -#### Parameter settings of the sg driver +This section is for person who already have a machine the LTFS is installed. Instruction how to use the LTFS is also available on [Wiki](https://github.com/LinearTapeFileSystem/ltfs/wiki). -LTFS uses the sg driver by default. You can improve reliability to change parameters of the sg driver below. +## Step1: List tape drives -``` -def_reserved_size=1048576 -``` +`# ltfs -o device_list` -In RHEL7, you can put following file as `/etc/modprobe.d/sg.conf`. +The output is like follows. You can have 3 drives in this example and you can use "Device Name" field, like `/dev/sg43` in this case, as the argument of ltfs command to mount the tape drive. ``` -options sg def_reserved_size=1048576 +50c4 LTFS14000I LTFS starting, LTFS version 2.4.0.0 (10022), log level 2. +50c4 LTFS14058I LTFS Format Specification version 2.4.0. +50c4 LTFS14104I Launched by "/home/piste/ltfsoss/bin/ltfs -o device_list". +50c4 LTFS14105I This binary is built for Linux (x86_64). +50c4 LTFS14106I GCC version is 4.8.5 20150623 (Red Hat 4.8.5-11). +50c4 LTFS17087I Kernel version: Linux version 3.10.0-514.10.2.el7.x86_64 (mockbuild@x86-039.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Mon Feb 20 02:37:52 EST 2017 i386. +50c4 LTFS17089I Distribution: NAME="Red Hat Enterprise Linux Server". +50c4 LTFS17089I Distribution: Red Hat Enterprise Linux Server release 7.3 (Maipo). +50c4 LTFS17089I Distribution: Red Hat Enterprise Linux Server release 7.3 (Maipo). +50c4 LTFS17085I Plugin: Loading "sg" tape backend. +Tape Device list:. +Device Name = /dev/sg43, Vender ID = IBM , Product ID = ULTRIUM-TD5 , Serial Number = 9A700L0077, Product Name = [ULTRIUM-TD5] . +Device Name = /dev/sg38, Vender ID = IBM , Product ID = ULT3580-TD6 , Serial Number = 00013B0119, Product Name = [ULT3580-TD6] . +Device Name = /dev/sg37, Vender ID = IBM , Product ID = ULT3580-TD7 , Serial Number = 00078D00C2, Product Name = [ULT3580-TD7] . ``` -A reboot is required for these settings to take effect. +## Step2: Format a tape -You can check current configuration of sg driver to see the file `/proc/scsi/sg/debug` like +As described into the LTFS format specifications, LTFS uses the partition feature of the tape drive. It means you can't use a tape just after you purchase a tape. You need format the tape before using int on LTFS. -``` -$ cat /proc/scsi/sg/debug -max_active_device=44 def_reserved_size=32768 - >>> device=sg25 1:0:10:0 em=0 sg_tablesize=1024 excl=0 open_cnt=1 - FD(1): timeout=60000ms bufflen=524288 (res)sgat=16 low_dma=0 - cmd_q=1 f_packid=0 k_orphan=0 closed=0 - No requests active - >>> device=sg26 1:0:10:1 em=0 sg_tablesize=1024 excl=0 open_cnt=1 - FD(1): timeout=60000ms bufflen=524288 (res)sgat=16 low_dma=0 - cmd_q=1 f_packid=0 k_orphan=0 closed=0 - No requests active -``` +To format a tape, you can use `mkltfs` command like -##### Performance improvement of the sg device +`# mkltfs -d 9A700L0077` -You can improve performance to change parameters of the sg driver below. But this option may cause I/O error reported as #144 in some HBAs. +In this case, `mkltfs` tries to format a tape in the tape drive `9A700L0077`. You can use a device name `/dev/sg43` instead. -``` -allow_dio=1 -``` +## Step3: Mount a tape through a tape drive -In RHEL7, you can put following file as `/etc/modprobe.d/sg.conf`. +After you prepared a formatted tape, you can mount it through a tape drive like -``` -options sg allow_dio=1 -``` +`# ltfs -o devname=9A700L0077 /ltfs` -A reboot is required for these settings to take effect. +In this command, the ltfs command try to mount the tape in the tape drive `9A700L0077` to `/ltfs` directory. Of cause, you can use a device name `/dev/sg43` instead. -At this time, we know following HBA's works correctly. +If mount process is successfully done, you can access to the LTFS tape through `/ltfs` directory. -- QLogic 8Gb FC HBAs +You must not touch any `st` devices while ltfs is mounting a tape. -And following HBA's doesn't work correctly. +## Step4: Unmount the tape drive -- ATTO ExpressSAS H680 -- Emulex FC HBAs (Some drivers work but some drivers dont work, see this [section](#note-for-the-lpfc-driver-emulex-fibre-hbas)) +You can use following command when you want to unmount the tape. The ltfs command try to write down the current meta-data to the tape and close the tape cleanly. -##### Note for the lpfc driver (Emulex Fibre HBAs) +`# umount /ltfs` -In the lpfc driver (for Emulex Fibre HBAs), the table size of the scatter-gather is 64 by default. This configuration may cause I/O errors intermittently when `allow_dio=1` is set and scatter-gather table cannot be reserved. To avoid this error, you need to change the parameter `lpfc_sg_seg_cnt` to 256 or greater like below. +One thing you need to pay attention here is it is not a unmount completion when umount command is returned. It just a finish of trigger to notify the unmount request to the ltfs command. Actual unmount is completed when the ltfs command is finished. -``` -options lpfc lpfc_sg_seg_cnt=256 -``` +## The `ltfsee_ordered_copy` utility -A reboot is required for these settings to take effect. +The [`ltfsee_ordered_copy`](https://github.com/LinearTapeFileSystem/ltfs/wiki/ltfs_ordered_copy) is a program to copy files from source to destination with LTFS order optimization. -In some versions of the lpfc driver (for Emulex Fibre HBAs), the table size of the scatter-gather cannot be changed correctly. You can check the value is changed or not in `sg_tablesize` value in `/proc/scsi/sg/debug`. If you don't have a correct value (256 or greater) in `sg_tablesize`, removing `allow_dio=1` configuration of the sg driver is strongly recommended. +It is written by python and it can work on both python2 and python3 (Python 2.7 or later is strongly recommended). You need to install the `pyxattr` module for both python2 and python3. -##### Note for buggy HBAs +# Building the LTFS from this GitHub project -LTFS doesn't support the HBAs which doesn't handle the transfer length of SCSI data by default. The reason is because the safety of the data but LTFS provides a option to relax this limitation. +These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. -You can use such kind of HBAs if you run the `configure` script with `--enable-buggy-ifs` option and build. +## Prerequisites for build -List of the HBA `--enable-buggy-ifs` is needed is below. +Please refer [this page](https://github.com/LinearTapeFileSystem/ltfs/wiki/Build-Environments). -[HBA list require `--enable-buggy-ifs`](https://github.com/LinearTapeFileSystem/ltfs/wiki/HBA-info) +## Build and install on Linux -#### IBM lin_tape driver support +``` +./autogen.sh +./configure +make +make install +``` -You need to add `--enable-lintape` as an argument of ./configure script if you want to build the backend for lin_tape. You also need to add `DEFAULT_TAPE=lin_tape` if you set the lin_tape backend as default backend. +`./configure --help` shows various options for build and install. -#### Buildable distributions +In some systems, you might need `sudo ldconfig -v` after `make install` to load the shared libraries correctly. - | Dist | Arch | Status | - |:-: |:-: |:-: | - | RHEL 8 | x86_64 | OK | - | RHEL 8 | ppc64le | OK | - | RHEL 7 | x86_64 | OK | - | RHEL 7 | ppc64le | OK | - | CentOS 8 | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS8%20Build%20Job/badge.svg?branch=master)| - | CentOS 8 | ppc64le | Probably OK | - | CentOS 7 | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS7%20Build%20Job/badge.svg?branch=master)| - | CentOS 7 | ppc64le | Probably OK | - | Fedora 28 | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Fedora28%20Build%20Job/badge.svg?branch=master)| - | Ubuntu 16.04 LTS | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2016.04%20Build%20Job/badge.svg?branch=master)| - | Ubuntu 16.04 LTS | ppc64le | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | - | Ubuntu 18.04 LTS | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2018.04%20Build%20Job/badge.svg?branch=master)| - | Ubuntu 18.04 LTS | ppc64le | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | - | Ubuntu 20.04 LTS (Need icu-config)| x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2020.04%20Build%20Job/badge.svg?branch=master)| - | Debian 9 | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian9%20Build%20Job/badge.svg?branch=master)| - | Debian 10 (Need icu-config) | x86_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian10%20Build%20Job/badge.svg?branch=master)| - | ArchLinux 2018.08.01 | x86_64 | Not checked automatically | - | ArchLinux 2018.12.31 (rolling) | x86_64 | Not checked automatically| +### Buildable Linux distributions + + | Dist | Arch | Status | + |:----------------------------------:|:-------:|:-------------------------------------------------------------------------------------------------------------------------------------:| + | RHEL 8 | x86\_64 | OK - Not checked automatically | + | RHEL 8 | ppc64le | OK - Not checked automatically | + | RHEL 7 | x86\_64 | OK - Not checked automatically | + | RHEL 7 | ppc64le | OK - Not checked automatically | + | CentOS 8 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS8%20Build%20Job/badge.svg?branch=master) | + | CentOS 8 | ppc64le | OK - Not checked automatically | + | CentOS 7 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS7%20Build%20Job/badge.svg?branch=master) | + | CentOS 7 | ppc64le | OK - Not checked automatically | + | Fedora 28 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Fedora28%20Build%20Job/badge.svg?branch=master) | + | Ubuntu 16.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2016.04%20Build%20Job/badge.svg?branch=master) | + | Ubuntu 16.04 LTS | ppc64le | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | + | Ubuntu 18.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2018.04%20Build%20Job/badge.svg?branch=master) | + | Ubuntu 18.04 LTS | ppc64le | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | + | Ubuntu 20.04 LTS (Need icu-config) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2020.04%20Build%20Job/badge.svg?branch=master) | + | Debian 9 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian9%20Build%20Job/badge.svg?branch=master) | + | Debian 10 (Need icu-config) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian10%20Build%20Job/badge.svg?branch=master) | + | ArchLinux 2018.08.01 | x86\_64 | OK - Not checked automatically | + | ArchLinux 2018.12.31 (rolling) | x86\_64 | OK - Not checked automatically | Currently, automatic build checking is working on GitHub Actions and Travis CI. For Ubuntu20.04 and Debian10, dummy `icu-config` is needed in the build machine. See Issue [#153](https://github.com/LinearTapeFileSystem/ltfs/issues/153). -### Build and install on OSX (macOS) +## Build and install on OSX (macOS) -#### Recent Homedrew system setup +### Recent Homedrew system setup Before build on macOS, you need to configure the environment like below. @@ -243,7 +173,7 @@ export PKG_CONFIG_PATH="/usr/local/opt/icu4c/lib/pkgconfig:/usr/local/opt/libxml export PATH="$PATH:$ICU_PATH:$LIBXML2_PATH" ``` -#### Old Homedrew system setup +### Old Homedrew system setup Before build on OSX (macOS), some include path adjustment is required. ``` @@ -251,7 +181,7 @@ brew link --force icu4c brew link --force libxml2 ``` -#### Building LTFS +### Building LTFS On OSX (macOS), snmp cannot be supported, you need to disable it on configure script. And may be, you need to specify LDFLAGS while running configure script to link some required frameworks, CoreFundation and IOKit. ``` @@ -263,13 +193,15 @@ make install `./configure --help` shows various options for build and install. -#### Buildable systems +#### Buildable macOS systems - | OS | Xcode | Package system | Status | - |:-: |:-: |:-: |:-: | - | macOS 10.14.6 | 11.3 | Homebrew | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | + | OS | Xcode | Package system | Status | + |:-------------:|:-----:|:--------------:|:-------------------------------------------------------------------------------------------------------------------------------------:| + | macOS 10.14.6 | 11.3 | Homebrew | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | + | macOS 10.15 | 12.4 | Homebrew | OK - Not checked automatically | + | macOS 11 | 12.4 | Homebrew | OK - Not checked automatically | -### Build and install on FreeBSD +## Build and install on FreeBSD Note that on FreeBSD, the usual 3rd party man directory is /usr/local/man. Configure defaults to using /usr/local/share/man. So, override it on the command line to avoid having man pages put in the wrong place. @@ -282,9 +214,10 @@ make install #### Buildable versions - | Version | Arch | Status | - |:-: |:-: |:-: | - | 11 | x86_64 | OK | + | Version | Arch | Status | + |:-------:|:-------:|:------------------------------:| + | 11 | x86\_64 | OK - Not checked automatically | + | 12 | x86\_64 | OK - Not checked automatically | ### Build and install on NetBSD @@ -297,11 +230,11 @@ make install #### Buildable versions - | Version | Arch | Status | - |:-: |:-: |:-: | - | 8.1 | amd64 | OK | - | 8.0 | i386 | OK | - | 7.2 | amd64 | OK | + | Version | Arch | Status | + |:-------:|:-----:|:------------------------------:| + | 8.1 | amd64 | OK - Not checked automatically | + | 8.0 | i386 | OK - Not checked automatically | + | 7.2 | amd64 | OK - Not checked automatically | ## Contributing From 0a51e3c2ccce9a75cb136c931013e5dc3cc188ae Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 10 May 2021 10:00:42 +0900 Subject: [PATCH 029/121] Handle 0-byte file on tape in `ltfs-ordered_copy` (#276) * Handle copy 0-byte file correctly * Fix the problem when only one file are specified as a source and `--keep-tree` is specified * Fix the destination path for copying EAs * Modify .gitignore --- .gitignore | 2 ++ src/utils/ltfs_ordered_copy | 6 ++++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index cae78d4d..0711b11e 100644 --- a/.gitignore +++ b/.gitignore @@ -23,6 +23,8 @@ ltfs.pc .libs/ messages/.lib .dirstamp +# Files created by OS +.DS_Store # Files for development GPATH GRTAGS diff --git a/src/utils/ltfs_ordered_copy b/src/utils/ltfs_ordered_copy index 10f14db8..900ec7d1 100755 --- a/src/utils/ltfs_ordered_copy +++ b/src/utils/ltfs_ordered_copy @@ -4,7 +4,7 @@ # OO_Copyright_BEGIN # # -# Copyright 2010, 2020 IBM Corp. All rights reserved. +# Copyright 2010, 2021 IBM Corp. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions @@ -72,6 +72,8 @@ class CopyItem: self.size = os.path.getsize(self.src) except Exception as e: self.logger.error('Failed to get attribute of "{0}": {1}'.format(self.src, str(str(e)))) + self.part = b'a' + self.start = 0 return (self.vuuid, self.part, self.start) @@ -92,7 +94,7 @@ class CopyItem: # Set EAs of the destination file (_, filename) = os.path.split(self.src) for key in src_attributes: - xattr.set(os.path.join(self.dst, filename), key, src_attributes[key]) + xattr.set(self.dst, key, src_attributes[key]) else: #Only copy data shutil.copy(self.src, self.dst) except Exception as e: From a949a124fa1e54f2e0d32ca45d7106f9fc5ed4df Mon Sep 17 00:00:00 2001 From: xloem <0xloem@gmail.com> Date: Mon, 10 May 2021 23:04:26 -0400 Subject: [PATCH 030/121] Wipe the plugin structure before loading a plugin. (#278) --- src/libltfs/plugin.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libltfs/plugin.c b/src/libltfs/plugin.c index bcf9f6bd..402a10ed 100644 --- a/src/libltfs/plugin.c +++ b/src/libltfs/plugin.c @@ -82,7 +82,7 @@ int plugin_load(struct libltfs_plugin *pl, const char *type, const char *name, CHECK_ARG_NULL(name, -LTFS_NULL_ARG); CHECK_ARG_NULL(config, -LTFS_NULL_ARG); - pl->lib_handle = NULL; + memset(pl, 0, sizeof(*pl)); lib_path = config_file_get_lib(type, name, config); if (! lib_path) { From cb7ce47b60b9aff1e93a9a02efe69223f4a0d712 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 11 May 2021 22:17:30 +0900 Subject: [PATCH 031/121] Update ISO status --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 2cc2507b..dfc90b38 100644 --- a/README.md +++ b/README.md @@ -44,10 +44,10 @@ The table below show status of the LTFS format Specification | Version | Status of SNIA | Status of ISO | |:-------:|:----------------------------------------------------------------------------------------------------------------------------------------:|:---------------:| - | 2.2 | [Published the Technical Position](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | Published | - | 2.3.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.PDF) | - | - | 2.4 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | Published | - | 2.5.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) | Not started yet | + | 2.2 | [Published the Technical Position](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | Published | + | 2.3.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.PDF) | - | + | 2.4 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | Published as `20919:2016` | + | 2.5.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) | Under development | # How to use the LTFS (Quick start) From f806d7014f559139b6989d6c4651555d8be7366e Mon Sep 17 00:00:00 2001 From: Ralf Zerres Date: Fri, 14 May 2021 03:22:44 +0200 Subject: [PATCH 032/121] Refresh obsoleted macros for newer autotools (#277) * Substitute depreciated macro `AC_TRY_COMPILE`, use prefered macro `AC_COMPILE_IFELSE` * Replace the obsoleted `AC_PROG_LIBTOOL` with `LT_INIT` Signed-off-by: Ralf Zerres --- configure.ac | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/configure.ac b/configure.ac index c2d66b28..e317546d 100644 --- a/configure.ac +++ b/configure.ac @@ -49,7 +49,7 @@ AC_USE_SYSTEM_EXTENSIONS AC_PROG_CC_C99 AM_PROG_CC_C_O AM_PROG_AR -AC_PROG_LIBTOOL +LT_INIT dnl dnl Detecting OS @@ -429,11 +429,12 @@ SAVE_CFLAGS=$CFLAGS CFLAGS="$CFLAGS ${LIBXML2_MODULE_CFLAGS}" AC_MSG_CHECKING(for XML_PARSE_HUGE) AC_CACHE_VAL(ac_cv_xml_huge, - AC_TRY_COMPILE([#include ], - [int foo = XML_PARSE_HUGE; return 0], - ac_cv_xml_huge=yes, - ac_cv_xml_huge=no - )) + AC_COMPILE_IFELSE( + [AC_LANG_PROGRAM([#include ], + [int foo = XML_PARSE_HUGE; return 0;])], + [ac_cv_xml_huge=yes], + [ac_cv_xml_huge=no] + )) AC_MSG_RESULT($ac_cv_xml_huge) CFLAGS="$SAVE_CFLAGS" From 55b0f8e4cc08b7ebe217b43be57c503a076a02d2 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 17 May 2021 11:48:27 +0900 Subject: [PATCH 033/121] Refactor the XML parser to return more graceful infomation (#281) * Handle errors in tape read call back in the XML parser * Refactor the XML parser to return the descriptive error code --- messages/internal_error/root.txt | 55 +++++ messages/libltfs/root.txt | 20 +- src/libltfs/arch/errormap.c | 54 ++++- src/libltfs/fs.c | 6 +- src/libltfs/ltfs.c | 3 +- src/libltfs/ltfs_error.h | 57 +++++ src/libltfs/ltfs_internal.c | 2 +- src/libltfs/xml.h | 198 +++++++++-------- src/libltfs/xml_reader.c | 56 +++-- src/libltfs/xml_reader_libltfs.c | 366 +++++++++++++++++++------------ 10 files changed, 555 insertions(+), 262 deletions(-) diff --git a/messages/internal_error/root.txt b/messages/internal_error/root.txt index 7fb6c8ae..cd9fafac 100644 --- a/messages/internal_error/root.txt +++ b/messages/internal_error/root.txt @@ -253,6 +253,61 @@ root:table { I1200E:string{ "The operation needs to be started again." } I1201E:string{ "Locate returns write-perm error." } I1202E:string{ "Failed to open a stats DB." } + I1203E:string{ "There is no trailing FM after an index." } + I1204E:string{ "Failed to update safename." } + + // Error codes for the XML parser + I5000E:string{ "Error happens in reading XML node." } + I5001E:string{ "Error happens in reading XML const value." } + I5002E:string{ "Unexpected node type is detected." } + I5003E:string{ "Unexpected XML document end is detected." } + I5004E:string{ "Cannot detect a tags is empty or not." } + I5005E:string{ "The tag is empty." } + I5006E:string{ "Error skipping unrecognized tag." } + I5007E:string{ "Do not find required tag." } + I5008E:string{ "Duplicated tag is detected." } + I5009E:string{ "XML tag is not closed correctly." } + I5010E:string{ "Cannot save the tag." } + I5011E:string{ "Unexpected top tag in the XML document." } + I5012E:string{ "Unexpected encoding is detected." } + I5013E:string{ "Failed to get attribute of the top tag." } + I5014E:string{ "Wrong UUID is detected." } + I5015E:string{ "Cannot parse generation number correctly." } + I5016E:string{ "Cannot parse index's updatetime correctly." } + I5017E:string{ "Wrong tape position of index or labal is detected." } + I5018E:string{ "Wrong tape position of previous index is detected." } + I5019E:string{ "Unexpected policyupdate value is detected." } + I5020E:string{ "Wrong policy value is detected." } + I5021E:string{ "Too long index comment." } + I5022E:string{ "Wrong maxuid tag is detected." } + I5023E:string{ "Unexpected readonly value is detected in a dir." } + I5024E:string{ "Unexpected timestamp in dir mtime." } + I5025E:string{ "Unexpected timestamp in dir creation time." } + I5026E:string{ "Unexpected timestamp in dir atime." } + I5027E:string{ "Unexpected timestamp in dir ctime." } + I5028E:string{ "Unexpected timestamp in dir backup time." } + I5029E:string{ "Unexpected xattr type." } + I5030E:string{ "Unexpected xattr size." } + I5031E:string{ "Unexpected UID number." } + I5032E:string{ "UID number validation is failed." } + I5033E:string{ "Unexpected readonly value is detected in a file." } + I5034E:string{ "Unexpected timestamp in file mtime." } + I5035E:string{ "Unexpected timestamp in file creation time." } + I5036E:string{ "Unexpected timestamp in file atime." } + I5037E:string{ "Unexpected timestamp in file ctime." } + I5038E:string{ "Unexpected timestamp in file backup time." } + I5039E:string{ "Unexpected file size value." } + I5040E:string{ "Unexpected partition ID." } + I5041E:string{ "Unexpected startblock." } + I5042E:string{ "Unexpected offset." } + I5043E:string{ "Unexpected bytecount." } + I5044E:string{ "Unexpected fileoffset." } + I5045E:string{ "Extent list is overlapped." } + I5046E:string{ "Extent list is longer than file size." } + I5047E:string{ "Unexpected timestamp in tape format time in a label." } + I5048E:string{ "Unexpected partition map in a label." } + I5049E:string{ "Unexpected blocksize in a label." } + I5050E:string{ "Unexpected compression in a label." } // Special error codes I9997E:string{ "Child process error (ltfsck/mkltfs): %s (%d)." } diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index dfe74acc..8dcd9708 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -559,13 +559,13 @@ v 17007E:string { "Cannot instantiate an LTFS label parser for file \'%s\'." } 17008E:string { "Cannot parse XML label from file \'%s\'." } 17009E:string { "Cannot instantiate an LTFS label parser for a memory buffer." } - 17010E:string { "Cannot parse XML label from memory." } + 17010E:string { "Cannot parse XML label from memory (%d)." } 17011E:string { "Cannot instantiate an index parser for file \'%s\'." } - 17012E:string { "Cannot parse index from file \'%s\'." } + 17012E:string { "Cannot parse index from file \'%s\' (%d)." } 17013E:string { "Cannot parse index: failed to determine medium position (%d)." } 17014E:string { "Cannot parse index: failed to create XML parser input buffer." } 17015E:string { "Cannot parse index: failed to create XML reader." } - 17016E:string { "Cannot parse index direct from medium." } + 17016E:string { "Cannot parse index direct from medium (%d)." } 17017E:string { "XML parser: unexpected top-level tag \'%s\'." } 17018E:string { "XML parser: unsupported encoding \'%s\'." } 17019E:string { "XML parser: no schema version found." } @@ -581,7 +581,7 @@ v 17029E:string { "XML parser: invalid UUID %s." } 17030E:string { "XML parser: failed to normalize %s \'%s\'." } 17031E:string { "XML parser: invalid %s \'%s\'." } - 17032E:string { "XML parser: compression must be \'true\' (1) or \'false\' (0)." } + 17032E:string { "XML parser: boolean must be \'true\' (1) or \'false\' (0) but it is \'%s\'." } 17033E:string { "XML parser: invalid partition \'%s\'." } 17034E:string { "XML parser: invalid time \'%s\' (%d)." } 17035E:string { "XML parser: expected a text node." } @@ -650,8 +650,8 @@ v 17098I:string { "Device Name = %s (%d.%d.%d.%d), Vendor ID = %s, Product ID = %s, Serial Number = %s, Product Name =%s." } 17099E:string { "Failed to spawn the periodic sync thread (%d)." } - 17100E:string { "XML parser: UID on the root directory must be 1." } - 17101E:string { "XML parser: UID 1 is reserved for the root directory." } + 17100E:string { "XML parser: UID on the root directory must be 1" } + 17101E:string { "XML parser: UID 1 is reserved for the root directory (%s)." } 17102E:string { "Cannot set PEWS: Mode Sense for Device Configuration Extension failed (%d)." } 17103E:string { "Cannot set PEWS: Mode Select for Device Configuration Extension failed (%d)." } 17104E:string { "Cannot get PEWS: Mode Sense for Device Configuration Extension failed (%d)." } @@ -810,6 +810,14 @@ v 17266I:string { "Skip setting the append only mode because the drive doesn't seem to support it." } 17267E:string { "Locate command returns write-perm error (%d). Replace a return code to %d." } + 17268E:string { "XML parser: Detected invalid %s in a label." } + 17269W:string { "XML parser: Failed to parse a key name of xattr in %s. Skip one xattr." } + 17270E:string { "XML parser: Detected invalid %s in a file (%s)." } + 17271E:string { "Cannot save the conflicted info of symlink and extent: file %s." } + 17272E:string { "XML parser: Detected invalid %s in a directory (%s)." } + 17273E:string { "XML parser: Detected an I/O error on reading an index from the tape (%d)." } + 17274W:string { "XML parser: Unexpected error code is detected (%d)." } + // For Debug 19999I:string { "%s %s %d." } // DO NOT place messages with IDs 20000 or higher here! diff --git a/src/libltfs/arch/errormap.c b/src/libltfs/arch/errormap.c index 4ee2fa3b..4d049c6d 100644 --- a/src/libltfs/arch/errormap.c +++ b/src/libltfs/arch/errormap.c @@ -288,7 +288,59 @@ static struct error_map fuse_error_list[] = { { LTFS_NEED_START_OVER, "I1200E", EINVAL}, { LTFS_LOCATE_ERROR, "I1201E", EIO}, { LTFS_STATS_DB_OPEN, "I1202E", EIO}, - + { LTFS_SAFENAME_FAIL, "I1204E", EINVAL}, + { LTFS_NO_TRAIL_FM, "I1203E", EINVAL}, + { LTFS_XML_READ_FAIL, "I5000E", EINVAL}, + { LTFS_XML_CONST_FAIL, "I5001E", EINVAL}, + { LTFS_XML_WRONG_NODE, "I5002E", EINVAL}, + { LTFS_XML_UNEXPECTED_EOF, "I5003E", EINVAL}, + { LTFS_XML_EMPTY_UNKNOWN, "I5004E", EINVAL}, + { LTFS_XML_EMPTY, "I5005E", EINVAL}, + { LTFS_XML_SKIP_FAIL, "I5006E", EINVAL}, + { LTFS_XML_NO_REQUIRED_TAG, "I5007E", EINVAL}, + { LTFS_XML_DUPLICATED_TAG, "I5008E", EINVAL}, + { LTFS_XML_OPEN_TAG, "I5009E", EINVAL}, + { LTFS_XML_SAVE_FAIL, "I5010E", EINVAL}, + { LTFS_XML_WRONG_TOPTAG, "I5011E", EINVAL}, + { LTFS_XML_WRONG_ENCODING, "I5012E", EINVAL}, + { LTFS_XML_TOP_ATTR_FAIL, "I5013E", EINVAL}, + { LTFS_XML_WRONG_UUID, "I5014E", EINVAL}, + { LTFS_XML_WRONG_GEN, "I5015E", EINVAL}, + { LTFS_XML_WRONG_UTIME, "I5016E", EINVAL}, + { LTFS_XML_WRONG_LOC, "I5017E", EINVAL}, + { LTFS_XML_WRONG_LOC_PREV, "I5018E", EINVAL}, + { LTFS_XML_WRONG_PA, "I5019E", EINVAL}, + { LTFS_XML_WRONG_POLICY, "I5020E", EINVAL}, + { LTFS_XML_TOO_LONG_COMMENT, "I5021E", EINVAL}, + { LTFS_XML_WRONG_NEXT, "I5022E", EINVAL}, + { LTFS_XML_WRONG_RO_DIR, "I5023E", EINVAL}, + { LTFS_XML_WRONG_MTIME_DIR, "I5024E", EINVAL}, + { LTFS_XML_WRONG_CRTIME_DIR, "I5025E", EINVAL}, + { LTFS_XML_WRONG_ATIME_DIR, "I5026E", EINVAL}, + { LTFS_XML_WRONG_CTIME_DIR, "I5027E", EINVAL}, + { LTFS_XML_WRONG_BTIME_DIR, "I5028E", EINVAL}, + { LTFS_XML_XATTR_TYPE, "I5029E", EINVAL}, + { LTFS_XML_XATTR_SIZE, "I5030E", EINVAL}, + { LTFS_XML_WRONG_UID, "I5031E", EINVAL}, + { LTFS_XML_INVALID_UID, "I5032E", EINVAL}, + { LTFS_XML_WRONG_RO_F, "I5033E", EINVAL}, + { LTFS_XML_WRONG_MTIME_F, "I5034E", EINVAL}, + { LTFS_XML_WRONG_CRTIME_F, "I5035E", EINVAL}, + { LTFS_XML_WRONG_ATIME_F, "I5036E", EINVAL}, + { LTFS_XML_WRONG_CTIME_F, "I5037E", EINVAL}, + { LTFS_XML_WRONG_BTIME_F, "I5038E", EINVAL}, + { LTFS_XML_WRONG_SIZE, "I5039E", EINVAL}, + { LTFS_XML_WRONG_PART, "I5040E", EINVAL}, + { LTFS_XML_WRONG_START_BLK, "I5041E", EINVAL}, + { LTFS_XML_WRONG_OFFSET, "I5042E", EINVAL}, + { LTFS_XML_WRONG_BYTE_CNT, "I5043E", EINVAL}, + { LTFS_XML_WRONG_FILE_OFST, "I5044E", EINVAL}, + { LTFS_XML_EXT_OVERLAP, "I5045E", EINVAL}, + { LTFS_XML_EXT_TOO_LONG, "I5046E", EINVAL}, + { LTFS_XML_WRONG_FTIME_L, "I5047E", EINVAL}, + { LTFS_XML_WRONG_PART_MAP, "I5048E", EINVAL}, + { LTFS_XML_WRONG_BLOCKSIZE, "I5049E", EINVAL}, + { LTFS_XML_WRONG_COMP, "I5050E", EINVAL}, { EDEV_NO_SENSE, "D0000E", EIO}, { EDEV_OVERRUN, "D0002E", EIO}, { EDEV_UNDERRUN, "D0003E", ENODATA}, diff --git a/src/libltfs/fs.c b/src/libltfs/fs.c index ab95a54f..01909508 100644 --- a/src/libltfs/fs.c +++ b/src/libltfs/fs.c @@ -845,7 +845,7 @@ static struct name_list* fs_update_platform_safe_names_and_hash_table(struct den int fs_update_platform_safe_names(struct dentry* basedir, struct ltfs_index *idx, struct name_list *list) { struct name_list *list_ptr, *list_tmp; - int rc = 0; + int ret = 0; list = fs_update_platform_safe_names_and_hash_table(basedir, idx, list, false, false); // normal loop list = fs_update_platform_safe_names_and_hash_table(basedir, idx, list, true, false); // add dup name @@ -857,12 +857,12 @@ int fs_update_platform_safe_names(struct dentry* basedir, struct ltfs_index *idx HASH_DEL(list, list_ptr); free(list_ptr); } - rc = -1; + ret = -LTFS_SAFENAME_FAIL; } HASH_CLEAR(hh, list); - return rc; + return ret; } /** diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index f19da62f..bfa9adf5 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1627,8 +1627,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec else ltfsmsg(LTFS_ERR, 11021E); /* read DP Index failed */ goto out_unlock; - } - else { + } else { ltfsmsg(LTFS_DEBUG, 11025D); /* volume is consistent */ vol->mount_type = MOUNT_ERR_TAPE; } diff --git a/src/libltfs/ltfs_error.h b/src/libltfs/ltfs_error.h index 7d158d9e..433ec15b 100644 --- a/src/libltfs/ltfs_error.h +++ b/src/libltfs/ltfs_error.h @@ -259,6 +259,63 @@ #define LTFS_NEED_START_OVER 1200 /* Need operation needs to be started again */ #define LTFS_LOCATE_ERROR 1201 /* Locate returns write-perm error */ #define LTFS_STATS_DB_OPEN 1202 /* Failed to open a stats DB */ +#define LTFS_NO_TRAIL_FM 1203 /* There is no trailing FM after an index */ +#define LTFS_SAFENAME_FAIL 1204 /* Failed to update safename */ + +/* + * Error codes for the XML parser + */ +#define LTFS_XML_READ_FAIL 5000 /* Error happens in reading XML node */ +#define LTFS_XML_CONST_FAIL 5001 /* Error happens in reading XML const value */ +#define LTFS_XML_WRONG_NODE 5002 /* Unexpected node type is detected */ +#define LTFS_XML_UNEXPECTED_EOF 5003 /* Unexpected XML document end is detected */ +#define LTFS_XML_EMPTY_UNKNOWN 5004 /* Cannot detect a tags is empty or not */ +#define LTFS_XML_EMPTY 5005 /* The tag is empty */ +#define LTFS_XML_SKIP_FAIL 5006 /* Error skipping unrecognized tag */ +#define LTFS_XML_NO_REQUIRED_TAG 5007 /* Do not find required tag */ +#define LTFS_XML_DUPLICATED_TAG 5008 /* Duplicated tag is detected */ +#define LTFS_XML_OPEN_TAG 5009 /* XML tag is not closed correctly */ +#define LTFS_XML_SAVE_FAIL 5010 /* Cannot save the tag */ +#define LTFS_XML_WRONG_TOPTAG 5011 /* Unexpected top tag in the XML document */ +#define LTFS_XML_WRONG_ENCODING 5012 /* Unexpected encoding is detected */ +#define LTFS_XML_TOP_ATTR_FAIL 5013 /* Failed to get attribute of the top tag */ +#define LTFS_XML_WRONG_UUID 5014 /* Wrong UUID is detected */ +#define LTFS_XML_WRONG_GEN 5015 /* Cannot parse generation number correctly */ +#define LTFS_XML_WRONG_UTIME 5016 /* Cannot parse index's updatetime correctly */ +#define LTFS_XML_WRONG_LOC 5017 /* Wrong tape position of index or labal is detected */ +#define LTFS_XML_WRONG_LOC_PREV 5018 /* Wrong tape position of previous index is detected */ +#define LTFS_XML_WRONG_PA 5019 /* Unexpected policyupdate value is detected */ +#define LTFS_XML_WRONG_POLICY 5020 /* Wrong policy value is detected */ +#define LTFS_XML_TOO_LONG_COMMENT 5021 /* Too long index comment */ +#define LTFS_XML_WRONG_NEXT 5022 /* Wrong maxuid tag is detected */ +#define LTFS_XML_WRONG_RO_DIR 5023 /* Unexpected readonly value is detected in a dir */ +#define LTFS_XML_WRONG_MTIME_DIR 5024 /* Unexpected timestamp in dir mtime */ +#define LTFS_XML_WRONG_CRTIME_DIR 5025 /* Unexpected timestamp in dir creation time */ +#define LTFS_XML_WRONG_ATIME_DIR 5026 /* Unexpected timestamp in dir atime */ +#define LTFS_XML_WRONG_CTIME_DIR 5027 /* Unexpected timestamp in dir ctime */ +#define LTFS_XML_WRONG_BTIME_DIR 5028 /* Unexpected timestamp in dir backup time */ +#define LTFS_XML_XATTR_TYPE 5029 /* Unexpected xattr type */ +#define LTFS_XML_XATTR_SIZE 5030 /* Unexpected xattr size */ +#define LTFS_XML_WRONG_UID 5031 /* Unexpected UID number */ +#define LTFS_XML_INVALID_UID 5032 /* UID number validation is failed */ +#define LTFS_XML_WRONG_RO_F 5033 /* Unexpected readonly value is detected in a file */ +#define LTFS_XML_WRONG_MTIME_F 5034 /* Unexpected timestamp in file mtime */ +#define LTFS_XML_WRONG_CRTIME_F 5035 /* Unexpected timestamp in file creation time */ +#define LTFS_XML_WRONG_ATIME_F 5036 /* Unexpected timestamp in file atime */ +#define LTFS_XML_WRONG_CTIME_F 5037 /* Unexpected timestamp in file ctime */ +#define LTFS_XML_WRONG_BTIME_F 5038 /* Unexpected timestamp in file backup time */ +#define LTFS_XML_WRONG_SIZE 5039 /* Unexpected file size value */ +#define LTFS_XML_WRONG_PART 5040 /* Unexpected partition ID */ +#define LTFS_XML_WRONG_START_BLK 5041 /* Unexpected startblock */ +#define LTFS_XML_WRONG_OFFSET 5042 /* Unexpected offset */ +#define LTFS_XML_WRONG_BYTE_CNT 5043 /* Unexpected bytecount */ +#define LTFS_XML_WRONG_FILE_OFST 5044 /* Unexpected fileoffset */ +#define LTFS_XML_EXT_OVERLAP 5045 /* Extent list is overlapped */ +#define LTFS_XML_EXT_TOO_LONG 5046 /* Extent list is longer than file size */ +#define LTFS_XML_WRONG_FTIME_L 5047 /* Unexpected timestamp in tape format time in a label */ +#define LTFS_XML_WRONG_PART_MAP 5048 /* Unexpected partition map in a label */ +#define LTFS_XML_WRONG_BLOCKSIZE 5049 /* Unexpected blocksize in a label */ +#define LTFS_XML_WRONG_COMP 5050 /* Unexpected compression in a label */ #define LTFS_ERR_MAX 19999 diff --git a/src/libltfs/ltfs_internal.c b/src/libltfs/ltfs_internal.c index 7257e031..f3d0055d 100644 --- a/src/libltfs/ltfs_internal.c +++ b/src/libltfs/ltfs_internal.c @@ -429,7 +429,7 @@ int ltfs_read_index(uint64_t eod_pos, bool recover_symlink, struct ltfs_volume * if (ret < 0) { ltfsmsg(LTFS_WARN, 11194W, ret); return ret; - } else if (ret == 1) + } else if (ret == LTFS_NO_TRAIL_FM) end_fm = false; /* check volume UUID */ diff --git a/src/libltfs/xml.h b/src/libltfs/xml.h index 14f1eab7..b81726f9 100644 --- a/src/libltfs/xml.h +++ b/src/libltfs/xml.h @@ -115,7 +115,7 @@ int xml_format_time(struct ltfs_timespec t, char** out); /* standard parser variables */ #define declare_parser(toptag) \ const char *name, *parent_tag = (toptag); \ - int i, type, empty; + int i, type, empty, ret; #define declare_parser_vars_noloop(toptag) \ const char *name, *value, *parent_tag = (toptag); \ @@ -123,11 +123,11 @@ int xml_format_time(struct ltfs_timespec t, char** out); #define declare_parser_vars(toptag) \ const char *name, *value, *parent_tag = (toptag); \ - int i, type, empty; + int i, type, empty, ret; #define declare_parser_vars_symlinknode(toptag) \ const char *name, *parent_tag = (toptag); \ - int type; + int type, ret; #define declare_parser_vars_symlink(toptag) \ const char *name, *parent_tag = (toptag); \ @@ -149,123 +149,141 @@ int xml_format_time(struct ltfs_timespec t, char** out); * NOTE: in order for break to work correctly, this macro is not wrapped in a do { ... } while (0) * loop. So be careful when using it! */ #define get_next_tag() \ - if (xml_next_tag(reader, parent_tag, &name, &type) < 0) \ - return -1; \ - if (type == XML_ELEMENT_DECL) \ + ret = xml_next_tag(reader, parent_tag, &name, &type); \ + if (ret < 0) \ + return ret; \ + if (type == XML_ELEMENT_DECL) \ break /* check standard tracking array for required tags which are not present. */ -#define check_required_tags() do { \ - for (i=0; i. */ -#define assert_not_empty() do { \ - empty = xmlTextReaderIsEmptyElement(reader); \ - if (empty < 0) { \ - ltfsmsg(LTFS_ERR, 17003E); \ - return -1; \ - } else if (empty > 0) { \ - ltfsmsg(LTFS_ERR, 17004E, name); \ - return -1; \ - } \ -} while (0) +#define assert_not_empty() \ + do { \ + empty = xmlTextReaderIsEmptyElement(reader); \ + if (empty < 0) { \ + ltfsmsg(LTFS_ERR, 17003E); \ + return -LTFS_XML_EMPTY_UNKNOWN; \ + } else if (empty > 0) { \ + ltfsmsg(LTFS_ERR, 17004E, name); \ + return -LTFS_XML_EMPTY; \ + } \ + } while (0) /* check whether a tag is empty. */ -#define check_empty() do { \ - empty = xmlTextReaderIsEmptyElement(reader); \ - if (empty < 0) { \ - ltfsmsg(LTFS_ERR, 17003E); \ - return -1; \ - } \ -} while (0) +#define check_empty() \ + do { \ + empty = xmlTextReaderIsEmptyElement(reader); \ + if (empty < 0) { \ + ltfsmsg(LTFS_ERR, 17003E); \ + return -LTFS_XML_EMPTY_UNKNOWN; \ + } \ + } while (0) /* consume the end of a tag, failing if there's extra content */ -#define check_tag_end(tagname) do { \ - if (xml_next_tag(reader, (tagname), &name, &type) < 0 || type != XML_ELEMENT_DECL) { \ - ltfsmsg(LTFS_ERR, 17005E, (tagname)); \ - return -1; \ - } \ -} while (0) +#define check_tag_end(tagname) \ + do { \ + if (xml_next_tag(reader, (tagname), &name, &type) < 0 || type != XML_ELEMENT_DECL) { \ + ltfsmsg(LTFS_ERR, 17005E, (tagname)); \ + return -LTFS_XML_OPEN_TAG; \ + } \ + } while (0) /* get text from a tag, failing if the tag is empty (like ) or contains * the empty string (like ). if successful, it * reads the text into "value". It does not consume the remainder of the tag. */ -#define get_tag_text() do { \ - assert_not_empty(); \ - if (xml_scan_text(reader, &value) < 0) \ - return -1; \ - if (strlen(value) == 0) { \ - ltfsmsg(LTFS_ERR, 17004E, name); \ - return -1; \ - } \ -} while (0) - -#define get_tag_text_allow_zero_length() do { \ - assert_not_empty(); \ - if (xml_scan_text(reader, &value) < 0) \ - return -1; \ -} while (0) +#define get_tag_text() \ + do { \ + assert_not_empty(); \ + ret = xml_scan_text(reader, &value); \ + if (ret < 0) \ + return ret; \ + if (strlen(value) == 0) { \ + ltfsmsg(LTFS_ERR, 17004E, name); \ + return -LTFS_XML_EMPTY; \ + } \ + } while (0) + +#define get_tag_text_allow_zero_length() \ + do { \ + assert_not_empty(); \ + ret = xml_scan_text(reader, &value); \ + if (ret < 0) \ + return ret; \ + } while (0) /* get text from a tag. if successful, it reads the text into "value". * It does not consume the remainder of the tag. */ -#define get_tag_text_allow_empty() do { \ - if (xml_scan_text(reader, &value) < 0) \ - return -1; \ -} while (0) +#define get_tag_text_allow_empty() \ + do { \ + ret = xml_scan_text(reader, &value); \ + if (ret < 0) { \ + return ret; \ + } \ + } while (0) /* issue a warning that the tag is unrecognized and will be ignored. */ -#define ignore_unrecognized_tag() do { \ - ltfsmsg(LTFS_WARN, 17006W, name, parent_tag); \ - if (xml_skip_tag(reader) < 0) \ - return -1; \ -} while (0) +#define ignore_unrecognized_tag() \ + do { \ + ltfsmsg(LTFS_WARN, 17006W, name, parent_tag); \ + if (xml_skip_tag(reader) < 0) \ + return -LTFS_XML_SKIP_FAIL; \ + } while (0) /* store a tag in a list of unrecognized tags, to be written back to tape later */ -#define preserve_unrecognized_tag(structure) do { \ - if (xml_save_tag(reader, &(structure)->tag_count, &(structure)->preserved_tags) < 0) \ - return -1; \ - if (xml_skip_tag(reader) < 0) \ - return -1; \ -} while (0) +#define preserve_unrecognized_tag(structure) \ + do { \ + ret = xml_save_tag(reader, &(structure)->tag_count, &(structure)->preserved_tags); \ + if (ret < 0) \ + return -LTFS_XML_SAVE_FAIL; \ + if (xml_skip_tag(reader) < 0) \ + return -LTFS_XML_SKIP_FAIL; \ + } while (0) /** * This structure is used to store state data when reading XML directly from tape using * the libxml2 I/O callback method. */ struct xml_input_tape { - struct ltfs_volume *vol; /**< LTFS volume to read */ - uint64_t current_pos; /**< Current block position of the drive. */ - uint64_t eod_pos; /**< EOD position of the current partition. */ - bool saw_small_block; /**< Have we seen a small block yet? */ - bool saw_file_mark; /**< If we saw a small blilock, was it a file mark? */ - char *buf; /**< 1-block input buffer. */ - uint32_t buf_size; /**< Input buffer size. */ - uint32_t buf_start; /**< Offset of first valid byte in input buffer. */ - uint32_t buf_used; /**< Current input buffer usage. */ + struct ltfs_volume *vol; /**< LTFS volume to read */ + int err_code; /**< Error code from tape backend */ + uint64_t current_pos; /**< Current block position of the drive. */ + uint64_t eod_pos; /**< EOD position of the current partition. */ + bool saw_small_block; /**< Have we seen a small block yet? */ + bool saw_file_mark; /**< If we saw a small blilock, was it a file mark? */ + char *buf; /**< 1-block input buffer. */ + uint32_t buf_size; /**< Input buffer size. */ + uint32_t buf_start; /**< Offset of first valid byte in input buffer. */ + uint32_t buf_used; /**< Current input buffer usage. */ }; int xml_input_tape_read_callback(void *context, char *buffer, int len); int xml_input_tape_close_callback(void *context); diff --git a/src/libltfs/xml_reader.c b/src/libltfs/xml_reader.c index 270c223c..6087e592 100644 --- a/src/libltfs/xml_reader.c +++ b/src/libltfs/xml_reader.c @@ -73,7 +73,7 @@ int xml_scan_text(xmlTextReaderPtr reader, const char **value) int type; if (xml_reader_read(reader) < 0) - return -1; + return -LTFS_XML_READ_FAIL; type = xmlTextReaderNodeType(reader); if (type == XML_ELEMENT_DECL) @@ -84,11 +84,11 @@ int xml_scan_text(xmlTextReaderPtr reader, const char **value) *value = (const char *)xmlTextReaderConstValue(reader); if (!(*value)) { ltfsmsg(LTFS_ERR, 17035E); - return -1; + return -LTFS_XML_CONST_FAIL; } } else { ltfsmsg(LTFS_ERR, 17036E, type); - return -1; + return -LTFS_XML_WRONG_NODE; } return 0; @@ -108,9 +108,11 @@ int xml_scan_text(xmlTextReaderPtr reader, const char **value) int xml_next_tag(xmlTextReaderPtr reader, const char *containing_name, const char **name, int *type) { + int ret; do { - if (xml_reader_read(reader) < 0) - return -1; + ret = xml_reader_read(reader); + if (ret < 0) + return ret; *name = (const char *)xmlTextReaderConstName(reader); *type = xmlTextReaderNodeType(reader); } while (strcmp(*name, containing_name) && (*type) != XML_ELEMENT_NODE); @@ -249,17 +251,19 @@ int xml_reader_read(xmlTextReaderPtr reader) int ret = xmlTextReaderRead(reader); if (ret < 0) { ltfsmsg(LTFS_ERR, 17037E); - return -1; + return -LTFS_XML_READ_FAIL; } else if (ret == 0) { ltfsmsg(LTFS_ERR, 17038E); - return -1; + return -LTFS_XML_UNEXPECTED_EOF; } return 0; } /** - * Parse a UUID from the tape into a provided buffer, converting to lower-case. The output buffer - * must be at least 37 bytes. + * Parse a UUID from the tape into a provided buffer, converting to lower-case. + * The output buffer must be at least 37. + * + * @return 0 on success or -1 on error (intentionally -1) */ int xml_parse_uuid(char *out_val, const char *val) { @@ -385,6 +389,8 @@ int xml_parse_ll(long long *out_val, const char *val) * Parse a positive base-10 unsigned long long from a string. * This function does not print an error message because it usually doesn't * have enough information to say something helpful. + * + * @return 0 on success or -1 on error (intentionally -1) */ int xml_parse_ull(unsigned long long *out_val, const char *val) { @@ -409,6 +415,8 @@ int xml_parse_ull(unsigned long long *out_val, const char *val) * Parse a positive base-16 unsigned long long from a string. * This function does not print an error message because it usually doesn't * have enough information to say something helpful. + * + * @return 0 on success or -1 on error (intentionally -1) */ int xml_parse_xll(unsigned long long *out_val, const char *val) { @@ -432,7 +440,8 @@ int xml_parse_xll(unsigned long long *out_val, const char *val) /** * Parse a boolean value from a string. Per the W3C boolean datatype, "true" and "1" return true * and "false" and "0" return false. - * @return 0 on success or a negative value on error. + * + * @return 0 on success or -1 on error (intentionally -1) */ int xml_parse_bool(bool *out_val, const char *value) { @@ -444,7 +453,7 @@ int xml_parse_bool(bool *out_val, const char *value) else if (! strcmp(value, "false") || ! strcmp(value, "0")) *out_val = false; else { - ltfsmsg(LTFS_ERR, 17032E); + ltfsmsg(LTFS_ERR, 17032E, value); return -1; } @@ -453,6 +462,8 @@ int xml_parse_bool(bool *out_val, const char *value) /** * Parse a time from the XML file into a timespec structure. + * + * @return 0 on success or -1 on error (intentionally -1) */ int xml_parse_time(bool msg, const char *fmt_time, struct ltfs_timespec *rawtime) { @@ -496,7 +507,7 @@ int xml_input_tape_read_callback(void *context, char *buffer, int len) struct xml_input_tape *ctx = context; ssize_t nread, nr2; char *buf2; - int bytes_saved, bytes_remaining; + int bytes_saved, bytes_remaining, ret_sp; if (len == 0) return 0; @@ -535,6 +546,7 @@ int xml_input_tape_read_callback(void *context, char *buffer, int len) if (nread < 0) { /* We know we're not at EOD, so read errors are unexpected. */ ltfsmsg(LTFS_ERR, 17039E, (int)nread); + ctx->err_code = nread; return -1; } else if ((size_t) nread < ctx->buf_size) { /* Caught a small read. If this is a file mark, position before it. If @@ -542,16 +554,19 @@ int xml_input_tape_read_callback(void *context, char *buffer, int len) ctx->saw_small_block = true; if (nread == 0) { ctx->saw_file_mark = true; - if (tape_spacefm(ctx->vol->device, -1) < 0) { + ret_sp = tape_spacefm(ctx->vol->device, -1); + if (ret_sp < 0) { ltfsmsg(LTFS_ERR, 17040E); + ctx->err_code = ret_sp; return -1; } } else if (ctx->eod_pos == 0 || (ctx->eod_pos > 0 && ctx->current_pos < ctx->eod_pos)) { /* Look for a trailing file mark. */ buf2 = malloc(ctx->vol->label->blocksize); - if (! buf2) { + if (!buf2) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + ctx->err_code = -LTFS_NO_MEMORY; return -1; } nr2 = tape_read(ctx->vol->device, buf2, ctx->vol->label->blocksize, false, @@ -560,11 +575,14 @@ int xml_input_tape_read_callback(void *context, char *buffer, int len) errno = 0; /* Clear errno because some OSs set errno in free() */ if (nr2 < 0) { /* Still not at EOD, so read errors are cause for alarm. */ ltfsmsg(LTFS_ERR, 17041E, (int)nr2); + ctx->err_code = nr2; return -1; } else if (nr2 == 0) { ctx->saw_file_mark = true; - if (tape_spacefm(ctx->vol->device, -1) < 0) { + ret_sp = tape_spacefm(ctx->vol->device, -1); + if (ret_sp < 0) { ltfsmsg(LTFS_ERR, 17040E); + ctx->err_code = ret_sp; return -1; } } @@ -589,14 +607,10 @@ int xml_input_tape_read_callback(void *context, char *buffer, int len) return len; } -/** - * Close callback for XML parser input using the libxml2 I/O routines. The input - * buffer should be empty at this point, so just free the parser context. +/** Close callback for XML parser input using the libxml2 I/O routines. */ int xml_input_tape_close_callback(void *context) { - struct xml_input_tape *ctx = context; - free(ctx->buf); - free(ctx); + /* Do nothing */ return 0; } diff --git a/src/libltfs/xml_reader_libltfs.c b/src/libltfs/xml_reader_libltfs.c index d4549b74..1a3b2be0 100644 --- a/src/libltfs/xml_reader_libltfs.c +++ b/src/libltfs/xml_reader_libltfs.c @@ -265,6 +265,8 @@ static int _xml_parse_nametype_allow_zero_length(xmlTextReaderPtr reader, struct /** * Verify that a given string really does represent a partition (single character, a-z). + * + * @return 0 on success or -1 on error (intentionally -1) */ static int _xml_parse_partition(const char *val) { @@ -326,27 +328,29 @@ static int _xml_parser_init(xmlTextReaderPtr reader, const char *top_name, int * { const char *name, *encoding; char *value; - int type, ver; + int type, ver, ret; + + ret = xml_next_tag(reader, "", &name, &type); + if (ret < 0) + return ret; - if (xml_next_tag(reader, "", &name, &type) < 0) - return -1; if (strcmp(name, top_name)) { ltfsmsg(LTFS_ERR, 17017E, name); - return -1; + return -LTFS_XML_WRONG_TOPTAG; } /* reject this XML file if it isn't UTF-8 */ encoding = (const char *)xmlTextReaderConstEncoding(reader); if (! encoding || strcmp(encoding, "UTF-8")) { ltfsmsg(LTFS_ERR, 17018E, encoding); - return -1; + return -LTFS_XML_WRONG_ENCODING; } /* check the version attribute of the top-level tag */ value = (char *)xmlTextReaderGetAttribute(reader, BAD_CAST "version"); if (! value) { ltfsmsg(LTFS_ERR, 17019E); - return -1; + return -LTFS_XML_TOP_ATTR_FAIL; } if (_xml_parse_version(value, &ver) < 0) { ltfsmsg(LTFS_ERR, 17020E, value); @@ -367,6 +371,8 @@ static int _xml_parser_init(xmlTextReaderPtr reader, const char *top_name, int * /** * Parse a partition location from a label. + * + * @return 0 on success or -1 on error (intentionally -1) */ static int _xml_parse_label_location(xmlTextReaderPtr reader, struct ltfs_label *label) { @@ -394,6 +400,8 @@ static int _xml_parse_label_location(xmlTextReaderPtr reader, struct ltfs_label /** * Parse a partition map from an XML file, storing it in the given label structure. + * + * @return 0 on success or -1 on error (intentionally -1) */ static int _xml_parse_partition_map(xmlTextReaderPtr reader, struct ltfs_label *label) { @@ -432,15 +440,15 @@ static int _xml_parse_partition_map(xmlTextReaderPtr reader, struct ltfs_label * */ static int _xml_parse_label(xmlTextReaderPtr reader, struct ltfs_label *label) { - int ret; unsigned long long value_int; declare_parser_vars("ltfslabel"); declare_tracking_arrays(7, 0); /* start the parser: find top-level "label" tag, check version and encoding */ - if (_xml_parser_init(reader, parent_tag, &label->version, - LTFS_LABEL_VERSION_MIN, LTFS_LABEL_VERSION_MAX) < 0) - return -1; + ret = _xml_parser_init(reader, parent_tag, &label->version, + LTFS_LABEL_VERSION_MIN, LTFS_LABEL_VERSION_MAX); + if (ret < 0) + return ret; /* parse label contents */ while (true) { @@ -454,7 +462,7 @@ static int _xml_parse_label(xmlTextReaderPtr reader, struct ltfs_label *label) label->creator = strdup(value); if (! label->creator) { ltfsmsg(LTFS_ERR, 10001E, name); - return -1; + return -LTFS_NO_MEMORY; } check_tag_end("creator"); @@ -462,37 +470,44 @@ static int _xml_parse_label(xmlTextReaderPtr reader, struct ltfs_label *label) check_required_tag(1); get_tag_text(); ret = xml_parse_time(true, value, &label->format_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17268E, "formattime"); + return -LTFS_XML_WRONG_FTIME_L; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17218W, "formattime", value); check_tag_end("formattime"); } else if (! strcmp(name, "volumeuuid")) { check_required_tag(2); get_tag_text(); - if (xml_parse_uuid(label->vol_uuid, value) < 0) - return -1; + if (xml_parse_uuid(label->vol_uuid, value) < 0) { + ltfsmsg(LTFS_ERR, 17268E, "volumeuuid"); + return -LTFS_XML_WRONG_UUID; + } check_tag_end("volumeuuid"); } else if (! strcmp(name, "location")) { check_required_tag(3); assert_not_empty(); - if (_xml_parse_label_location(reader, label) < 0) - return -1; + if (_xml_parse_label_location(reader, label) < 0) { + ltfsmsg(LTFS_ERR, 17268E, "location"); + return -LTFS_XML_WRONG_LOC; + } } else if (! strcmp(name, "partitions")) { check_required_tag(4); assert_not_empty(); - if (_xml_parse_partition_map(reader, label) < 0) - return -1; + if (_xml_parse_partition_map(reader, label) < 0) { + ltfsmsg(LTFS_ERR, 17268E, "partitions"); + return -LTFS_XML_WRONG_PART_MAP; + } } else if (! strcmp(name, "blocksize")) { check_required_tag(5); get_tag_text(); if (xml_parse_ull(&value_int, value) < 0 || value_int == 0) { ltfsmsg(LTFS_ERR, 17022E, value); - return -1; + return -LTFS_XML_WRONG_BLOCKSIZE; } label->blocksize = value_int; check_tag_end("blocksize"); @@ -500,8 +515,10 @@ static int _xml_parse_label(xmlTextReaderPtr reader, struct ltfs_label *label) } else if (! strcmp(name, "compression")) { check_required_tag(6); get_tag_text(); - if (xml_parse_bool(&label->enable_compression, value) < 0) - return -1; + if (xml_parse_bool(&label->enable_compression, value) < 0) { + ltfsmsg(LTFS_ERR, 17268E, "compression"); + return -LTFS_XML_WRONG_COMP; + } check_tag_end("compression"); } else @@ -539,7 +556,7 @@ static int _xml_parse_ip_criteria(xmlTextReaderPtr reader, struct ltfs_index *id get_tag_text(); if (xml_parse_ull(&value_int, value) < 0) { ltfsmsg(LTFS_ERR, 17024E, value); - return -1; + return -LTFS_XML_WRONG_POLICY; } idx->original_criteria.max_filesize_criteria = value_int; check_tag_end("size"); @@ -570,7 +587,7 @@ static int _xml_parse_ip_criteria(xmlTextReaderPtr reader, struct ltfs_index *id if (index_criteria_dup_rules(&idx->index_criteria, &idx->original_criteria) < 0) { /* Could not duplicate index criteria rules */ ltfsmsg(LTFS_ERR, 11301E); - return -1; + return -LTFS_NO_MEMORY; } check_required_tags(); @@ -592,8 +609,9 @@ static int _xml_parse_policy(xmlTextReaderPtr reader, struct ltfs_index *idx) if (! strcmp(name, "indexpartitioncriteria")) { check_required_tag(0); assert_not_empty(); - if (_xml_parse_ip_criteria(reader, idx) < 0) - return -1; + ret = _xml_parse_ip_criteria(reader, idx); + if (ret < 0) + return ret; } else ignore_unrecognized_tag(); @@ -616,7 +634,7 @@ static int _xml_parse_one_extent(xmlTextReaderPtr reader, int idx_version, struc xt = calloc(1, sizeof(struct extent_info)); if (!xt) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); - return -1; + return -LTFS_NO_MEMORY; } while (true) { @@ -627,7 +645,7 @@ static int _xml_parse_one_extent(xmlTextReaderPtr reader, int idx_version, struc get_tag_text(); if (_xml_parse_partition(value) < 0) { free(xt); - return -1; + return -LTFS_XML_WRONG_PART; } xt->start.partition = value[0]; check_tag_end("partition"); @@ -637,7 +655,7 @@ static int _xml_parse_one_extent(xmlTextReaderPtr reader, int idx_version, struc get_tag_text(); if (xml_parse_ull(&value_int, value) < 0) { free(xt); - return -1; + return -LTFS_XML_WRONG_START_BLK; } xt->start.block = value_int; check_tag_end("startblock"); @@ -647,7 +665,7 @@ static int _xml_parse_one_extent(xmlTextReaderPtr reader, int idx_version, struc get_tag_text(); if (xml_parse_ull(&value_int, value) < 0) { free(xt); - return -1; + return -LTFS_XML_WRONG_OFFSET; } xt->byteoffset = value_int; check_tag_end("byteoffset"); @@ -657,7 +675,7 @@ static int _xml_parse_one_extent(xmlTextReaderPtr reader, int idx_version, struc get_tag_text(); if (xml_parse_ull(&value_int, value) < 0) { free(xt); - return -1; + return -LTFS_XML_WRONG_BYTE_CNT; } xt->bytecount = value_int; check_tag_end("bytecount"); @@ -667,7 +685,7 @@ static int _xml_parse_one_extent(xmlTextReaderPtr reader, int idx_version, struc get_tag_text(); if (xml_parse_ull(&value_int, value) < 0) { free(xt); - return -1; + return -LTFS_XML_WRONG_FILE_OFST; } xt->fileoffset = value_int; check_tag_end("fileoffset"); @@ -704,7 +722,7 @@ static int _xml_parse_one_extent(xmlTextReaderPtr reader, int idx_version, struc /* Overlap error */ ltfsmsg(LTFS_ERR, 17097E); free(xt); - return -1; + return -LTFS_XML_EXT_OVERLAP; } } if (! xt_used) @@ -735,8 +753,9 @@ static int _xml_parse_extents(xmlTextReaderPtr reader, int idx_version, struct d if (! strcmp(name, "extent")) { assert_not_empty(); - if (_xml_parse_one_extent(reader, idx_version, d) < 0) - return -1; + ret = _xml_parse_one_extent(reader, idx_version, d); + if (ret < 0) + return ret; } else ignore_unrecognized_tag(); @@ -759,7 +778,7 @@ static int _xml_parse_one_xattr(xmlTextReaderPtr reader, struct dentry *d) xattr = calloc(1, sizeof(struct xattr_info)); if (! xattr) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); - return -1; + return -LTFS_NO_MEMORY; } while (true) { @@ -769,8 +788,11 @@ static int _xml_parse_one_xattr(xmlTextReaderPtr reader, struct dentry *d) check_required_tag(0); /* Allow slash in xattr key */ - if (_xml_parse_nametype(reader, &xattr->key, true) < 0) + ret = _xml_parse_nametype(reader, &xattr->key, true); + if (ret < 0) { + ltfsmsg(LTFS_WARN, 17269W, d->name.name); free(xattr); + } check_tag_end("key"); @@ -781,35 +803,35 @@ static int _xml_parse_one_xattr(xmlTextReaderPtr reader, struct dentry *d) if (xattr_type && strcmp(xattr_type, "text") && strcmp(xattr_type, "base64")) { ltfsmsg(LTFS_ERR, 17027E, xattr_type); free(xattr); - return -1; + return -LTFS_XML_XATTR_TYPE; } check_empty(); if (empty == 0) { - if (xml_scan_text(reader, &value) < 0) { + ret = xml_scan_text(reader, &value); + if (ret < 0) { free(xattr->key.name); free(xattr); - return -1; + return ret; } + if (! xattr_type || ! strcmp(xattr_type, "text")) { xattr->value = strdup(value); if (! xattr->value) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); free(xattr->key.name); free(xattr); - return -1; + return -LTFS_NO_MEMORY; } xattr->size = strlen(value); } else { /* base64 */ - xattr->size = base64_decode( - (const unsigned char *)value, - strlen(value), - (unsigned char **)(&xattr->value)); + xattr->size = base64_decode((const unsigned char *)value, strlen(value), + (unsigned char **)(&xattr->value)); if (xattr->size == 0) { ltfsmsg(LTFS_ERR, 17028E); free(xattr->key.name); free(xattr); - return -1; + return -LTFS_XML_XATTR_SIZE; } } if (strlen(value) > 0) @@ -850,8 +872,9 @@ static int _xml_parse_xattrs(xmlTextReaderPtr reader, struct dentry *d) if (! strcmp(name, "xattr")) { assert_not_empty(); - if (_xml_parse_one_xattr(reader, d) < 0) - return -1; + ret = _xml_parse_one_xattr(reader, d); + if (ret < 0) + return ret; } else ignore_unrecognized_tag(); @@ -863,12 +886,13 @@ static int _xml_parse_xattrs(xmlTextReaderPtr reader, struct dentry *d) /** * Parse a tape offset (a partition tag and a startblock tag) from an XML source. + * * @param reader the XML source * @param tag name of the tag containing the tape position. This function will read to the end * of that tag, so it is not suitable for reading an extent list (which has other * children that need to be parsed). * @param pos pointer to a structure where the offset will be stored - * @return 0 on success or a negative value on error + * @return 0 on success or -1 on error (intentionally -1) */ static int _xml_parse_tapepos(xmlTextReaderPtr reader, const char *tag, struct tape_offset *pos) { @@ -914,7 +938,7 @@ static int _xml_save_symlink_conflict( struct ltfs_index *idx, struct dentry *d) err_d = realloc( idx->symlink_conflict, c * sizeof(size_t)); if (! err_d) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); - return -1; + return -LTFS_NO_MEMORY; } err_d[c-1] = d; @@ -927,9 +951,9 @@ static int _xml_save_symlink_conflict( struct ltfs_index *idx, struct dentry *d) /** * Parse a file into the given directory. */ -static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, struct dentry *dir, struct name_list *filename) +static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, struct dentry *dir, + struct name_list *filename) { - int ret; unsigned long long value_int; struct dentry *file; struct extent_info *xt_last; @@ -940,7 +964,7 @@ static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, stru file = fs_allocate_dentry(dir, NULL, NULL, false, false, false, idx); if (! file) { ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); - return -1; + return -LTFS_NO_MEMORY; } while (true) { @@ -962,25 +986,30 @@ static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, stru } else if (!strcmp(name, "length")) { check_required_tag(1); get_tag_text(); - if (xml_parse_ull(&value_int, value) < 0) - return -1; + if (xml_parse_ull(&value_int, value) < 0) { + ltfsmsg(LTFS_ERR, 17270E, "length", file->name.name); + return -LTFS_XML_WRONG_SIZE; + } file->size = value_int; check_tag_end("length"); } else if (!strcmp(name, "readonly")) { check_required_tag(2); get_tag_text(); - if (xml_parse_bool(&file->readonly, value) < 0) - return -1; + if (xml_parse_bool(&file->readonly, value) < 0) { + ltfsmsg(LTFS_ERR, 17270E, "readonly", file->name.name); + return -LTFS_XML_WRONG_RO_F; + } check_tag_end("readonly"); } else if (!strcmp(name, "modifytime")) { check_required_tag(3); get_tag_text(); ret = xml_parse_time(true, value, &file->modify_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17270E, "modifytime", file->name.name); + return -LTFS_XML_WRONG_MTIME_F; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "modifytime", file->name.name, (unsigned long long)file->uid, value); check_tag_end("modifytime"); @@ -988,9 +1017,10 @@ static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, stru check_required_tag(4); get_tag_text(); ret = xml_parse_time(true, value, &file->creation_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17270E, "creationtime", file->name.name); + return -LTFS_XML_WRONG_CRTIME_F; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "creationtime", file->name.name, (unsigned long long)file->uid, value); check_tag_end("creationtime"); @@ -999,9 +1029,10 @@ static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, stru check_required_tag(5); get_tag_text(); ret = xml_parse_time(true, value, &file->access_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17270E, "accesstime", file->name.name); + return -LTFS_XML_WRONG_ATIME_F; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "accesstime", file->name.name, (unsigned long long)file->uid, value); check_tag_end("accesstime"); @@ -1009,31 +1040,44 @@ static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, stru check_required_tag(6); get_tag_text(); ret = xml_parse_time(true, value, &file->change_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17270E, "changetime", file->name.name); + return -LTFS_XML_WRONG_CTIME_F; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "changetime", file->name.name, (unsigned long long)file->uid, value); check_tag_end("changetime"); } else if (!strcmp(name, "extendedattributes")) { check_optional_tag(0); check_empty(); - if (empty == 0 && _xml_parse_xattrs(reader, file) < 0) - return -1; + + if (empty == 0) { + ret = _xml_parse_xattrs(reader, file); + if(ret < 0) { + ltfsmsg(LTFS_ERR, 17270E, "extendedattributes", file->name.name); + return ret; + } + } } else if (!strcmp(name, "extentinfo")) { check_optional_tag(1); check_empty(); - if (empty == 0 && _xml_parse_extents(reader, idx->version, file) < 0) - return -1; - else extent_flag = true; + if (empty == 0) { + ret = _xml_parse_extents(reader, idx->version, file); + if(ret < 0) { + ltfsmsg(LTFS_ERR, 17270E, "extentinfo", file->name.name); + return ret; + } + } else extent_flag = true; } else if (!strcmp(name, "symlink")) { check_optional_tag(2); - if (_xml_parse_nametype(reader, &file->target, true) < 0) { + ret = _xml_parse_nametype(reader, &file->target, true); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17270E, "symlink", file->name.name); free(file); - return -1; + return ret; } file->isslink = true; @@ -1055,8 +1099,10 @@ static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, stru } else if (idx->version >= IDX_VERSION_UID && ! strcmp(name, UID_TAGNAME)) { check_required_tag(7); get_tag_text(); - if (xml_parse_ull(&value_int, value) < 0) - return -1; + if (xml_parse_ull(&value_int, value) < 0) { + ltfsmsg(LTFS_ERR, 17270E, UID_TAGNAME, file->name.name); + return -LTFS_XML_WRONG_UID; + } file->uid = value_int; if (file->uid > idx->uid_number) idx->uid_number = file->uid; @@ -1069,9 +1115,10 @@ static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, stru check_required_tag(8); get_tag_text(); ret = xml_parse_time(true, value, &file->backup_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17270E, BACKUPTIME_TAGNAME, file->name.name); + return -LTFS_XML_WRONG_BTIME_F; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "backuptime", file->name.name, (unsigned long long)file->uid, value); check_tag_end(BACKUPTIME_TAGNAME); @@ -1104,19 +1151,23 @@ static int _xml_parse_file(xmlTextReaderPtr reader, struct ltfs_index *idx, stru xt_last = TAILQ_LAST(&file->extentlist, extent_struct); if (xt_last->fileoffset + xt_last->bytecount > file->size) { ltfsmsg(LTFS_ERR, 17026E); - return -1; + return -LTFS_XML_EXT_TOO_LONG; } } /* Validate UID: must be nonzero (UID 0 is reserved for the root directory) */ if (file->uid == 0) { - ltfsmsg(LTFS_ERR, 17101E); - return -1; + ltfsmsg(LTFS_ERR, 17101E, file->name.name); + return -LTFS_XML_WRONG_UID; } if ( symlink_flag && extent_flag ) { ltfsmsg(LTFS_ERR, 17180E, file->name.name); - if ( _xml_save_symlink_conflict( idx, file ) ) return -1; + ret = _xml_save_symlink_conflict(idx, file); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17271E, file->name.name); + return ret; + } } return 0; @@ -1132,7 +1183,6 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, static int _xml_parse_dir_contents(xmlTextReaderPtr reader, struct dentry *dir, struct ltfs_index *idx) { - int ret = 0; struct name_list *list = NULL, *entry_name = NULL; CHECK_ARG_NULL(dir, -LTFS_NULL_ARG); declare_parser("contents"); @@ -1173,6 +1223,7 @@ static int _xml_parse_dir_contents(xmlTextReaderPtr reader, struct dentry *dir, ignore_unrecognized_tag(); entry_name = NULL; } + if (!strcmp(name, "file") || (!strcmp(name, "directory") && !(!dir && idx->root))) { /* Make temporal hash table whose key is dentry name */ HASH_ADD_KEYPTR(hh, list, entry_name->name, strlen(entry_name->name), entry_name); @@ -1199,8 +1250,9 @@ static int _xml_parse_dir_contents(xmlTextReaderPtr reader, struct dentry *dir, The file or directory of which name contains invalid char is skipped in this step. After that update platfrom_safe_name regarding skipped file or directory as the second step. These steps make a prioritization of name mangling.*/ - if (fs_update_platform_safe_names(dir, idx, list)!=0) { - return -1; + ret = fs_update_platform_safe_names(dir, idx, list); + if (ret < 0) { + return ret; } check_required_tags(); @@ -1220,7 +1272,6 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, struct ltfs_index *idx, struct ltfs_volume *vol, struct name_list *dirname) { - int ret; unsigned long long value_int; struct dentry *dir; @@ -1252,6 +1303,7 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, if (parent) { ret = _xml_parse_nametype(reader, &dir->name, false); if (ret < 0) { + ltfsmsg(LTFS_ERR, 17272E, "name", parent->name.name); free(dir); return ret; } @@ -1269,8 +1321,10 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, idx->volume_name.name = NULL; } else { ret = _xml_parse_nametype_allow_zero_length(reader, &idx->volume_name, false); - if (ret < 0) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17272E, "name", "/"); return ret; + } if (idx->volume_name.name) check_tag_end("name"); @@ -1280,17 +1334,20 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, } else if (!strcmp(name, "readonly")) { check_required_tag(1); get_tag_text(); - if (xml_parse_bool(&dir->readonly, value) < 0) - return -1; + if (xml_parse_bool(&dir->readonly, value) < 0) { + ltfsmsg(LTFS_ERR, 17272E, "readonly", dir->name.name); + return -LTFS_XML_WRONG_RO_DIR; + } check_tag_end("readonly"); } else if (!strcmp(name, "modifytime")) { check_required_tag(2); get_tag_text(); ret = xml_parse_time(true, value, &dir->modify_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17272E, "modifytime", dir->name.name); + return -LTFS_XML_WRONG_MTIME_DIR; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "updatetime", dir->name.name, (unsigned long long)dir->uid, value); check_tag_end("modifytime"); @@ -1299,9 +1356,10 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, check_required_tag(3); get_tag_text(); ret = xml_parse_time(true, value, &dir->creation_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17272E, "creationtime", dir->name.name); + return -LTFS_XML_WRONG_CRTIME_DIR; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "creationtime", dir->name.name, (unsigned long long)dir->uid, value); check_tag_end("creationtime"); @@ -1310,9 +1368,10 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, check_required_tag(4); get_tag_text(); ret = xml_parse_time(true, value, &dir->access_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17272E, "accesstime", dir->name.name); + return -LTFS_XML_WRONG_CTIME_DIR; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "accesstime", dir->name.name, (unsigned long long)dir->uid, value); check_tag_end("accesstime"); @@ -1321,14 +1380,15 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, check_required_tag(5); get_tag_text(); ret = xml_parse_time(true, value, &dir->change_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17272E, "changetime", dir->name.name); + return -LTFS_XML_WRONG_CTIME_DIR; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "changetime", dir->name.name, (unsigned long long)dir->uid, value); check_tag_end("changetime"); - } else if (! strcmp(name, "contents")) { + } else if (!strcmp(name, "contents")) { check_required_tag(6); check_empty(); if (empty == 0) { @@ -1340,14 +1400,21 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, } else if (!strcmp(name, "extendedattributes")) { check_optional_tag(0); check_empty(); - if (empty == 0 && _xml_parse_xattrs(reader, dir) < 0) - return -1; + if (empty == 0) { + ret = _xml_parse_xattrs(reader, dir); + if(ret < 0) { + ltfsmsg(LTFS_ERR, 17272E, "extendedattributes", dir->name.name); + return ret; + } + } } else if (idx->version >= IDX_VERSION_UID && ! strcmp(name, UID_TAGNAME)) { check_required_tag(7); get_tag_text(); - if (xml_parse_ull(&value_int, value) < 0) - return -1; + if (xml_parse_ull(&value_int, value) < 0) { + ltfsmsg(LTFS_ERR, 17272E, UID_TAGNAME, dir->name.name); + return -LTFS_XML_WRONG_UID; + } dir->uid = value_int; if (dir->uid > idx->uid_number) idx->uid_number = dir->uid; @@ -1362,9 +1429,10 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, check_required_tag(8); get_tag_text(); ret = xml_parse_time(true, value, &dir->backup_time); - if (ret < 0) - return -1; - else if (ret == LTFS_TIME_OUT_OF_RANGE) + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17272E, BACKUPTIME_TAGNAME, dir->name.name); + return -LTFS_XML_WRONG_BTIME_DIR; + } else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17220W, "backuptime", dir->name.name, (unsigned long long)dir->uid, value); check_tag_end(BACKUPTIME_TAGNAME); @@ -1398,14 +1466,14 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, /* Validate UID: root directory must have uid==1, other dentries must have nonzero UID */ /* TODO: would be nice to verify that there are no UID conflicts */ if (parent && dir->uid == 1) { - ltfsmsg(LTFS_ERR, 17101E); - return -1; + ltfsmsg(LTFS_ERR, 17101E, dir->name.name); + return -LTFS_XML_INVALID_UID; } else if (! parent && dir->uid != 1) { ltfsmsg(LTFS_ERR, 17100E); - return -1; + return -LTFS_XML_INVALID_UID; } else if (dir->uid == 0) { ltfsmsg(LTFS_ERR, 17106E); - return -1; + return -LTFS_XML_INVALID_UID; } return 0; @@ -1421,7 +1489,6 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, */ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, struct ltfs_volume *vol) { - int ret; unsigned long long value_int; declare_parser_vars("ltfsindex"); declare_tracking_arrays(8, 4); @@ -1468,7 +1535,7 @@ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, st idx->creator = strdup(value); if (! idx->creator) { ltfsmsg(LTFS_ERR, 10001E, name); - return -1; + return -LTFS_NO_MEMORY; } check_tag_end("creator"); @@ -1476,7 +1543,7 @@ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, st check_required_tag(1); get_tag_text(); if (xml_parse_uuid(idx->vol_uuid, value) < 0) - return -1; + return -LTFS_XML_WRONG_UUID; check_tag_end("volumeuuid"); } else if (! strcmp(name, "generationnumber")) { @@ -1484,7 +1551,7 @@ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, st get_tag_text(); if (xml_parse_ull(&value_int, value) < 0) { ltfsmsg(LTFS_ERR, 17023E, value); - return -1; + return -LTFS_XML_WRONG_GEN; } idx->generation = value_int; check_tag_end("generationnumber"); @@ -1494,7 +1561,7 @@ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, st get_tag_text(); ret = xml_parse_time(true, value, &idx->mod_time); if (ret < 0) - return -1; + return -LTFS_XML_WRONG_UTIME; else if (ret == LTFS_TIME_OUT_OF_RANGE) ltfsmsg(LTFS_WARN, 17219W, "updatetime", value); @@ -1504,13 +1571,13 @@ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, st check_required_tag(4); assert_not_empty(); if (_xml_parse_tapepos(reader, "location", &idx->selfptr) < 0) - return -1; + return -LTFS_XML_WRONG_LOC; } else if (! strcmp(name, "allowpolicyupdate")) { check_required_tag(5); get_tag_text(); if (xml_parse_bool(&idx->criteria_allow_update, value) < 0) - return -1; + return -LTFS_XML_WRONG_PA; check_tag_end("allowpolicyupdate"); } else if (! strcmp(name, "directory")) { @@ -1524,27 +1591,29 @@ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, st check_optional_tag(0); assert_not_empty(); if (_xml_parse_tapepos(reader, "previousgenerationlocation", &idx->backptr) < 0) - return -1; + return -LTFS_XML_WRONG_LOC_PREV; } else if (! strcmp(name, "dataplacementpolicy")) { check_optional_tag(1); assert_not_empty(); - if (_xml_parse_policy(reader, idx) < 0) - return -1; + ret = _xml_parse_policy(reader, idx); + if (ret < 0) + return ret; } else if (! strcmp(name, "comment")) { check_optional_tag(2); get_tag_text(); if (strlen(value) > INDEX_MAX_COMMENT_LEN) { ltfsmsg(LTFS_ERR, 17094E); - return -1; + return -LTFS_XML_TOO_LONG_COMMENT; } idx->commit_message = strdup(value); if (! idx->commit_message) { ltfsmsg(LTFS_ERR, 10001E, "_xml_parse_schema: index comment"); - return -1; + return -LTFS_NO_MEMORY; } check_tag_end("comment"); + } else if (! strcmp(name, "volumelockstate")) { check_optional_tag(3); get_tag_text(); @@ -1557,14 +1626,16 @@ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, st idx->vollock = PERMLOCKED_MAM; } check_tag_end("volumelockstate"); + } else if (idx->version >= IDX_VERSION_UID && ! strcmp(name, NEXTUID_TAGNAME)) { check_required_tag(7); get_tag_text(); if (xml_parse_ull(&value_int, value) < 0) - return -1; + return -LTFS_XML_WRONG_NEXT; if (value_int > idx->uid_number) idx->uid_number = value_int; check_tag_end(NEXTUID_TAGNAME); + } else if (! strcmp(name, NEXTUID_TAGNAME)) { ignore_unrecognized_tag(); @@ -1598,8 +1669,9 @@ static int _xml_parse_symlink_target(xmlTextReaderPtr reader, int idx_version, s if (! strcmp(name, "target")) { d->isslink = true; - if (_xml_parse_nametype(reader, &d->target, true) < 0) - return -1; + ret = _xml_parse_nametype(reader, &d->target, true); + if (ret < 0) + return ret; } else ignore_unrecognized_tag(); } @@ -1737,7 +1809,8 @@ int xml_label_from_mem(const char *buf, int buf_size, struct ltfs_label *label) ret = _xml_parse_label(reader, label); if (ret < 0) { - ltfsmsg(LTFS_ERR, 17010E); + /* TODO: Update the message */ + ltfsmsg(LTFS_ERR, 17010E, ret); ret = -LTFS_LABEL_INVALID; } xmlFreeTextReader(reader); @@ -1774,7 +1847,7 @@ int xml_schema_from_file(const char *filename, struct ltfs_index *idx, struct lt doc = xmlTextReaderCurrentDoc(reader); ret = _xml_parse_schema(reader, idx, vol); if (ret < 0) - ltfsmsg(LTFS_ERR, 17012E, filename); + ltfsmsg(LTFS_ERR, 17012E, filename, ret); if (doc) xmlFreeDoc(doc); xmlFreeTextReader(reader); @@ -1829,6 +1902,7 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) return -LTFS_NO_MEMORY; } ctx->vol = vol; + ctx->err_code = 0; ctx->current_pos = current_pos.block; ctx->eod_pos = eod_pos; ctx->saw_small_block = false; @@ -1853,6 +1927,8 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) if (! reader) { ltfsmsg(LTFS_ERR, 17015E); xmlFreeParserInputBuffer(read_buf); + free(ctx->buf); + free(ctx); return -LTFS_LIBXML2_FAILURE; } @@ -1863,6 +1939,8 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) ltfsmsg(LTFS_ERR, 17015E); xmlFreeTextReader(reader); xmlFreeParserInputBuffer(read_buf); + free(ctx->buf); + free(ctx); return -LTFS_LIBXML2_FAILURE; } #endif @@ -1873,21 +1951,33 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) /* Generate the Index. */ ret = _xml_parse_schema(reader, vol->index, vol); + if (ctx->err_code < 0) { + /* Error happens while reading tape */ + ltfsmsg(LTFS_ERR, 17273E, ctx->err_code); + ret = ctx->err_code; + } if (ret < 0) { - ltfsmsg(LTFS_ERR, 17016E); + ltfsmsg(LTFS_ERR, 17016E, ret); if (ret == -1) { - /* TODO: Need to return more descriptive error codes */ + /* Unexpected error code, we need to investigate */ + ltfsmsg(LTFS_ERR, 17274W, ret); ret = -LTFS_INDEX_INVALID; } } else if (ret == 0) { - if( ! ctx->saw_file_mark) - ret = 1; + if(!ctx->saw_file_mark) { + /* Return positive value intentionally, for recovering later */ + ret = LTFS_NO_TRAIL_FM; + } } + if (doc) xmlFreeDoc(doc); xmlFreeTextReader(reader); xmlFreeParserInputBuffer(read_buf); + free(ctx->buf); + free(ctx); + #ifdef DEBUG /* dump the tree if it isn't too large */ if (ret >= 0 && vol->index->file_count < 1000) From f4cf6f36b51c1daab211e8ec62c65a23e2e79a3b Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 17 May 2021 13:22:56 +0900 Subject: [PATCH 034/121] Modify the description for the spec --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index dfc90b38..d4ffeb87 100644 --- a/README.md +++ b/README.md @@ -43,11 +43,11 @@ LTFS Format Specification is specified data placement, shape of index and names The table below show status of the LTFS format Specification | Version | Status of SNIA | Status of ISO | - |:-------:|:----------------------------------------------------------------------------------------------------------------------------------------:|:---------------:| - | 2.2 | [Published the Technical Position](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | Published | - | 2.3.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.PDF) | - | - | 2.4 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | Published as `20919:2016` | - | 2.5.1 | [Published the Technical Position](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_v2.5_Technical_Position.pdf) | Under development | + |:-------:|:------------------------------------------------------------------------------------------------------------:|:-------------------------:| + | 2.2 | [Published](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | Published as `20919:2016` | + | 2.3.1 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.PDF) | - | + | 2.4 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | - | + | 2.5.1 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2%205%201_Standard.pdf) | Under development | # How to use the LTFS (Quick start) From b2a5b50666213dcf7b91750b7da9a2e19d9c370a Mon Sep 17 00:00:00 2001 From: piste2750 <32239919+piste2750@users.noreply.github.com> Date: Thu, 20 May 2021 11:01:11 +0900 Subject: [PATCH 035/121] Fix a build break on FreeBSD (#284) --- src/tape_drivers/freebsd/cam/cam_cmn.h | 6 +++--- src/tape_drivers/freebsd/cam/cam_tc.c | 6 ++++-- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/src/tape_drivers/freebsd/cam/cam_cmn.h b/src/tape_drivers/freebsd/cam/cam_cmn.h index 5b3a1afd..f56fe1dd 100644 --- a/src/tape_drivers/freebsd/cam/cam_cmn.h +++ b/src/tape_drivers/freebsd/cam/cam_cmn.h @@ -451,8 +451,8 @@ extern int camtape_get_info(void *device, struct tc_drive_info *info); extern int camtape_set_profiler(void *device, char *work_dir, bool enable); extern int camtape_get_timeout(struct timeout_tape *table, int op_code); -extern int camtape_logsense_page(struct camtape_data *softc, const uint8_t page, const uint8_t subpage, - unsigned char *buf, const size_t size); +extern int camtape_logsense(void *device, const uint8_t page, const uint8_t subpage, + unsigned char *buf, const size_t size); /** * Get Dump @@ -579,7 +579,7 @@ static inline bool is_dump_required_error(struct camtape_data *softc, int ret, b log_page.parm_pointer = 0; memset(log_page.data, 0, LOGSENSEPAGE); - rc = camtape_logsense_page(softc, log_page.page_code, 0, (unsigned char *)&log_page.data, + rc = camtape_logsense(softc, log_page.page_code, 0, (unsigned char *)&log_page.data, LOGSENSEPAGE); ans = (rc == DEVICE_GOOD); diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 168db00a..db75aea8 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -1139,7 +1139,7 @@ int camtape_load(void *device, struct tc_position *pos) /* * Non-IBM drive doesn't have cartridge type so need to assume from density code. */ - softc->cart_type = assume_cart_type(priv->density_code); + softc->cart_type = assume_cart_type(softc->density_code); if (buf[2] == 0x01) softc->is_worm = true; } else { @@ -1686,7 +1686,7 @@ int camtape_format(void *device, TC_FORMAT_TYPE format, const char *vol_name, co */ #define MAX_UINT16 (0x0000FFFF) -int camtape_logsense(struct camtape_data *softc, const uint8_t page, const uint8_t subpage, +int camtape_logsense(void *device, const uint8_t page, const uint8_t subpage, unsigned char *buf, const size_t size) { int rc = DEVICE_GOOD; @@ -1695,6 +1695,8 @@ int camtape_logsense(struct camtape_data *softc, const uint8_t page, const uint8 union ccb *ccb = NULL; int timeout; + struct camtape_data *softc = device; + unsigned int len = 0; unsigned char *inner_buf = NULL; From d08cb4404bf2936f9debda994a299bf8f82b13dd Mon Sep 17 00:00:00 2001 From: y-sasn <75974685+y-sasn@users.noreply.github.com> Date: Thu, 27 May 2021 15:32:10 +0900 Subject: [PATCH 036/121] Introduce Recommended access order (RAO) via EA (#288) Co-authored-by: y-sasaki --- messages/libltfs/root.txt | 9 +- src/libltfs/ltfs.c | 145 +++++++++++ src/libltfs/ltfs.h | 12 + src/libltfs/ltfs_fsops.c | 13 +- src/libltfs/tape.c | 46 +++- src/libltfs/tape.h | 12 + src/libltfs/tape_ops.h | 21 ++ src/libltfs/xattr.c | 13 +- src/tape_drivers/freebsd/cam/cam_tc.c | 20 ++ src/tape_drivers/generic/file/filedebug_tc.c | 20 ++ src/tape_drivers/generic/itdtimg/itdtimg_tc.c | 20 ++ .../linux/lin_tape/lin_tape_ibmtape.c | 20 ++ src/tape_drivers/linux/sg/sg_tape.c | 246 +++++++++++++++++- .../netbsd/scsipi-ibmtape/scsipi_ibmtape.c | 20 ++ src/tape_drivers/osx/iokit/iokit_tape.c | 20 ++ 15 files changed, 627 insertions(+), 10 deletions(-) diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index 8dcd9708..e177b796 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -809,7 +809,6 @@ v 17265I:string { "Skip writing the index because of %s." } 17266I:string { "Skip setting the append only mode because the drive doesn't seem to support it." } 17267E:string { "Locate command returns write-perm error (%d). Replace a return code to %d." } - 17268E:string { "XML parser: Detected invalid %s in a label." } 17269W:string { "XML parser: Failed to parse a key name of xattr in %s. Skip one xattr." } 17270E:string { "XML parser: Detected invalid %s in a file (%s)." } @@ -817,6 +816,14 @@ v 17272E:string { "XML parser: Detected invalid %s in a directory (%s)." } 17273E:string { "XML parser: Detected an I/O error on reading an index from the tape (%d)." } 17274W:string { "XML parser: Unexpected error code is detected (%d)." } + 17275E:string { "Failed to get supported uds size from drive." } + 17276E:string { "Too many files to process on this drive. Valid size: max_uds_supported: %u, max_uds_size: %u." } + 17277D:string { "RAO list clear: %s." } + 17278E:string { "RAO command returns error on %s (%d)." } + 17279E:string { "File number exceeds the maximum limit for RAO. Maximum file supported: %u, given: %u." } + 17280E:string { "Error occured when reading file set for RAO: %s." } + 17281E:string { "Failed to create RAO output file: %s." } + 17282E:string { "Failed to read RAO input file: %s." } // For Debug 19999I:string { "%s %s %d." } diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index bfa9adf5..e6442150 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -4227,3 +4227,148 @@ int ltfs_profiler_set(uint64_t source, struct ltfs_volume *vol) return ret; } + +static int _ltfs_write_rao_file(char *write_data, char *file_path, size_t *write_size) +{ + int rc = -EDEV_UNKNOWN; + char *path; + + rc = asprintf(&path, "%s%s", file_path, LTFS_OUT_FILE_EXTENSION); + if (rc < 0) { + ltfsmsg(LTFS_ERR, 10001E, __FILE__); + return -LTFS_NO_MEMORY; + } + + FILE *p; + p = fopen(path, "wb"); + if (!p) { + ltfsmsg(LTFS_ERR, 17281E, "File open failed"); + return -LTFS_FILE_ERR; + } else { + fwrite(&write_data, sizeof(char), (size_t)write_size, p); + fclose(p); + rc = DEVICE_GOOD; + } + + return rc; +} + +static int _ltfs_read_rao_file(char *read_data, uint32_t *num_of_files, char *file_path) +{ + int rc = -EDEV_UNKNOWN; + char *path; + int size = 0; + + rc = asprintf(&path, "%s", file_path); + if (rc < 0) { + ltfsmsg(LTFS_ERR, 10001E, __FILE__); + return -LTFS_NO_MEMORY; + } + + /* file reader */ + FILE *p; + p = fopen(path, "rb"); + if (!p) { + ltfsmsg(LTFS_ERR, 17282E, "File open failed"); + return -EDEV_INVALID_ARG; + } + + rc = fseeko(p, 0LL, SEEK_END); + if(rc < 0){ + ltfsmsg(LTFS_ERR, 17282E, "File seek failed"); + return -LTFS_FILE_ERR; + } + size = ftello(p); + + if (size >= RAO_MAX_RET_SIZE) { + ltfsmsg(LTFS_ERR, 17282E, "File size is too big"); + return -EDEV_INVALID_ARG; + } else if (size < 0) { + ltfsmsg(LTFS_ERR, 17282E, "File seek failed"); + return -LTFS_FILE_ERR; + } + + rc = fseeko(p,0,SEEK_SET); + if (rc < 0) { + ltfsmsg(LTFS_ERR, 17282E, "File seek failed"); + return -LTFS_FILE_ERR; + } + + rc = fread(read_data, sizeof(char), size, p); + fclose(p); + if (rc < 0) { + ltfsmsg(LTFS_ERR, 17282E, "File read failed"); + return -LTFS_FILE_ERR; + } + + /* Input parser, counts how many files (num_of_files) there are in the input. */ + /* The grao param list size should be Header(8 bytes) + ( UDS(32 bytes) * num_of_files ). */ + if ( (size - 8) % 32 != 0 || size < 8+32 ) { + /* data format is wrong */ + ltfsmsg(LTFS_ERR, 17282E, "Unreadable file format"); + return -EDEV_INVALID_ARG; + } + *num_of_files = (size - 8) / 32; + + return rc; +} + +/** + * Perform the RAO commands and save the results in a .out file. + * @param path The file location of the GRAO buffer to handle. + * @param vol LTFS volume. The label structure receives a new UUID and format time; all other + * label fields should be filled in correctly before calling this function. + */ +int ltfs_get_rao_list(char *path, struct ltfs_volume *vol) +{ + int ret = -EDEV_UNKNOWN; + ret = tape_device_lock(vol->device); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 12010E, __FUNCTION__); + return ret; + } + + /* Prepare memory and read data from file */ + struct rao_mod rao; + vol->device->rao = &rao; + char in_buf[RAO_MAX_RET_SIZE]; /* read bytes is never larger than return size */ + rao.in_buf = (char *)&in_buf; + + if (path == NULL){ + ret = -EDEV_INVALID_ARG; + goto out; + } + + /* get rao */ + uint32_t *tmp_num = &rao.num_of_files; + ret = _ltfs_read_rao_file(rao.in_buf, tmp_num, path); + if (ret < 0) { + goto out; + } + + /* file size check */ + if (rao.num_of_files >= RAO_MAX_FILENUM) { + ltfsmsg(LTFS_ERR, 17279E, RAO_MAX_FILENUM, rao.num_of_files); + ret = -EDEV_INVALID_ARG; + goto out; + } + + /* Send RAO request*/ + ret = tape_rao_request(vol->device, vol->device->rao); + if (ret < 0) { + goto out; + } + + /* write data to file */ + ret = _ltfs_write_rao_file(rao.out_buf, path, rao.out_size); + if (ret < 0) { + goto out; + } + + goto out; + +out: + tape_device_unlock(vol->device); + vol->device->rao = NULL; + return ret; +} \ No newline at end of file diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index f0e8bb9c..d2eb61b2 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -168,6 +168,14 @@ struct device_data; #define LTFS_LIVELINK_EA_NAME "ltfs.vendor.IBM.prefixLength" +/* rao parameters */ +#define LTFS_OUT_FILE_EXTENSION ".out" +#define RAO_MAX_FILENUM 2700 /* Maximum file number allowed for RAO */ +#define LTFS_GEOMETORY_OFF (0x00) +#define LTFS_GEOMETORY_ON (0x01) + /* Maximum size allowed that is returned from RAO */ +#define RAO_MAX_RET_SIZE RAO_MAX_FILENUM * (32 + LTFS_GEOMETORY_OFF*20) + 8 + #define INTERRUPTED_GOTO(rc, label) \ do{ \ if (ltfs_is_interrupted()) { \ @@ -440,6 +448,7 @@ struct ltfs_volume { int file_open_count; /**< Number of opened files */ const char *work_directory; + }; struct ltfs_label { @@ -698,6 +707,9 @@ int ltfs_print_device_list(struct tape_ops *ops); void ltfs_enable_livelink_mode(struct ltfs_volume *vol); int ltfs_profiler_set(uint64_t source, struct ltfs_volume *vol); + +int ltfs_get_rao_list(char *path, struct ltfs_volume *vol); + #ifdef __cplusplus } #endif diff --git a/src/libltfs/ltfs_fsops.c b/src/libltfs/ltfs_fsops.c index 671b2cfe..e426dbfb 100644 --- a/src/libltfs/ltfs_fsops.c +++ b/src/libltfs/ltfs_fsops.c @@ -1082,11 +1082,11 @@ int ltfs_fsops_setxattr(const char *path, const char *name, const char *value, s id->uid = d->uid; id->ino = d->ino; - /* Save original value */ - ret_restore = xattr_get(d, new_name_strip, value_restore, sizeof(value_restore), vol); - - ret = xattr_set(d, new_name_strip, value, size, flags, vol); if (dcache_initialized(vol)) { + /* Save original value */ + ret_restore = xattr_get(d, new_name_strip, value_restore, sizeof(value_restore), vol); + + ret = xattr_set(d, new_name_strip, value, size, flags, vol); if (ret == 0) { ret = dcache_setxattr(new_path, d, new_name_strip, value, size, flags, vol); if (ret < 0) { @@ -1097,9 +1097,10 @@ int ltfs_fsops_setxattr(const char *path, const char *name, const char *value, s } } dcache_close(d, true, true, vol); - } else + } else { + ret = xattr_set(d, new_name_strip, value, size, flags, vol); fs_release_dentry(d); - + } if (NEED_REVAL(ret)) { ret = ltfs_revalidate(write_lock, vol); if (ret == 0) diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index c2017385..cffaaebe 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -3559,7 +3559,7 @@ int tape_is_reformattable(struct device_data *dev, unsigned char cart_type, unsi * Enable profiler function * @param device a pointer to the tape device * @param work_dir work directory to store profiler data - * @paran enable enable or disable profiler function of this backend + * @param enable enable or disable profiler function of this backend * @return 0 on success or a negative value on error */ int tape_set_profiler(struct device_data *dev, char *work_dir, bool enable) @@ -3570,3 +3570,47 @@ int tape_set_profiler(struct device_data *dev, char *work_dir, bool enable) return dev->backend->set_profiler(dev->backend_data, work_dir, enable); } + +/** + * Prepares GRAO parameter list, and sends rao request. + * All data should be set in rao before this is called. + * @param dev a pointer to the tape device + * @param rao a pointer to the rao data + * @param [out] ret_buf the returned buffer from grao. + * @return 0 on success or a negative value on error + */ +int tape_rao_request(struct device_data *dev, struct rao_mod *rao) +{ + int ret = 0; + + /* check file size */ + if (rao->num_of_files <= 0) { + /* rao list will be cleared if call size is less than zero */ + rao->in_buf = NULL; + ltfsmsg(LTFS_DEBUG, 17277D, "Clear Called"); + } + + if (rao->in_buf == NULL){ + return EDEV_INTERNAL_ERROR; + } + + /* run GRAO */ + ret = dev->backend->grao(dev->backend_data, (const unsigned char *)rao->in_buf, rao->num_of_files); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17278E, "GRAO", ret); //GRAO command returns error + return ret; + } else if (rao->num_of_files <= 0) { + /* rao list cleared */ + ltfsmsg(LTFS_DEBUG, 17277D, "Clear Done"); + return ret; + } + + /* run RRAO */ + ret = dev->backend->rrao(dev->backend_data, rao->num_of_files, (char *)&rao->out_buf, (size_t *)&rao->out_size); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17278E, "RRAO", ret); + return ret; + } + + return ret; +} \ No newline at end of file diff --git a/src/libltfs/tape.h b/src/libltfs/tape.h index c202a92b..6020ff8f 100644 --- a/src/libltfs/tape.h +++ b/src/libltfs/tape.h @@ -75,6 +75,8 @@ extern "C" { #define MB (KB * 1024) #define GB (MB * 1024) +#define COMMAND_DESCRIPTION_LENGTH (32) + #define NEED_REVAL(ret) (ret == -EDEV_POR_OR_BUS_RESET \ || ret == -EDEV_MEDIUM_MAY_BE_CHANGED \ || ret == -EDEV_RESERVATION_PREEMPTED \ @@ -84,6 +86,13 @@ extern "C" { #define IS_UNEXPECTED_MOVE(ret) (ret == -EDEV_MEDIUM_REMOVAL_REQ) +struct rao_mod { + uint32_t num_of_files; /* number of files to process */ + char *in_buf; /* buffer to set in grao */ + char *out_buf; /* buffer returned from rrao */ + size_t *out_size; /* buffer size returned in out_buf */ +}; + struct device_data { struct tc_position position; /**< Current head position */ tape_block_t append_pos[2]; /**< Append positions, 0 means append at EOD */ @@ -106,6 +115,8 @@ struct device_data { ltfs_mutex_t backend_mutex; /**< Mutex to control backend access */ ltfs_mutex_t read_only_flag_mutex; /**< Mutex to control read_only access */ char *serial_number; /**< Serial number for identification */ + + struct rao_mod *rao; /**< RAO related module */ }; int tape_device_alloc(struct device_data **device); @@ -228,6 +239,7 @@ int tape_is_mountable(struct device_data *dev, char *barcode, unsigned char cart_type, unsigned char density); int tape_is_reformattable(struct device_data *dev, unsigned char cart_type, unsigned char density); int tape_set_profiler(struct device_data *dev, char *work_dir, bool enable); +int tape_rao_request(struct device_data *dev, struct rao_mod *rao); static inline char* tape_get_serialnumber(struct device_data *dev) { diff --git a/src/libltfs/tape_ops.h b/src/libltfs/tape_ops.h index b64b7ebb..6a1af0e6 100644 --- a/src/libltfs/tape_ops.h +++ b/src/libltfs/tape_ops.h @@ -927,6 +927,27 @@ struct tape_ops { * @param device Device handle returned by the backend's open(). */ bool (*is_readonly)(void *device); + + /** + * Submit the GRAO (Generate Recommended Access Order) command. + * The generated result can be called from the RRAO command. + * @param device A pointer to the tape device + * @param buf buffer data to process + * @param num_of_files The number of files to process for RAO. + * @return 0 on success or a negative value on error + */ + int (*grao)(void *device, const unsigned char *buf, const uint32_t num_of_files); + + /** + * Submit the RRAO (Receive Recommended Access Order) command. + * This should be called after the GRAO command. + * @param device A pointer to the tape device + * @param[out] out_buf rrao, rao result in binary. You need to parse it by yourself. + * @param[out] out_size size of out_buf that is retuned. + * @return 0 on success or a negative value on error + */ + int (*rrao)(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size); + }; /** diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index 76c711b9..40281028 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -805,6 +805,7 @@ bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_volume *v || ! strcmp(name, "ltfs.vendor.IBM.cartridgeMountNode") || ! strcmp(name, "ltfs.vendor.IBM.logLevel") || ! strcmp(name, "ltfs.vendor.IBM.syslogLevel") + || ! strcmp(name, "ltfs.vendor.IBM.rao") || ! strcmp(name, "ltfs.vendor.IBM.logPage") || ! strcmp(name, "ltfs.vendor.IBM.mediaMAM") || ! strncmp(name, "ltfs.vendor", strlen("ltfs.vendor"))) @@ -1387,6 +1388,16 @@ int _xattr_set_virtual(struct dentry *d, const char *name, const char *value, } else ret = -LTFS_STRING_CONVERSION; free(v); + } else if (! strcmp(name, "ltfs.vendor.IBM.rao")) { + char *v; + v = strndup(value, size); + if (! v) { + ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + return -LTFS_NO_MEMORY; + } + if (strlen(v) > PATH_MAX) return -LTFS_LARGE_XATTR; /* file path size check */ + ret = ltfs_get_rao_list(v, vol); + free(v); } else if (! strcmp(name, "ltfs.vendor.IBM.trace")) { char *v; @@ -1772,4 +1783,4 @@ bool _xattr_is_worm_ea(const char *name) return true; } return false; -} +} \ No newline at end of file diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index db75aea8..856bf094 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -2311,6 +2311,24 @@ int camtape_allow_overwrite(void *device, const struct tc_position pos) return rc; } +/** + * GRAO command is currently unsupported on this device + */ +int camtape_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + +/** + * RRAO command is currently unsupported on this device + */ +int camtape_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + /** * Set compression setting * @param device a pointer to the camtape backend @@ -4135,6 +4153,8 @@ struct tape_ops camtape_drive_handler = { .write_attribute = camtape_write_attribute, .read_attribute = camtape_read_attribute, .allow_overwrite = camtape_allow_overwrite, + .grao = camtape_grao, + .rrao = camtape_rrao, // May be command combination .set_compression = camtape_set_compression, .set_default = camtape_set_default, diff --git a/src/tape_drivers/generic/file/filedebug_tc.c b/src/tape_drivers/generic/file/filedebug_tc.c index 10191c3c..fa934d1f 100644 --- a/src/tape_drivers/generic/file/filedebug_tc.c +++ b/src/tape_drivers/generic/file/filedebug_tc.c @@ -2111,6 +2111,24 @@ int filedebug_allow_overwrite(void *device, const struct tc_position pos) return DEVICE_GOOD; } +/** + * GRAO command is currently unsupported on this device + */ +int filedebug_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + +/** + * RRAO command is currently unsupported on this device + */ +int filedebug_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + int filedebug_get_eod_status(void *device, int partition) { struct filedebug_data *state = (struct filedebug_data *)device; @@ -2796,6 +2814,8 @@ struct tape_ops filedebug_handler = { .write_attribute = filedebug_write_attribute, .read_attribute = filedebug_read_attribute, .allow_overwrite = filedebug_allow_overwrite, + .grao = filedebug_grao, + .rrao = filedebug_rrao, .set_compression = filedebug_set_compression, .set_default = filedebug_set_default, .get_cartridge_health = filedebug_get_cartridge_health, diff --git a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c index 16625a18..5809dd47 100644 --- a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c +++ b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c @@ -962,6 +962,24 @@ int itdtimage_allow_overwrite(void *device, const struct tc_position pos) return DEVICE_GOOD; } +/** + * GRAO command is currently unsupported on this device + */ +int itdtimage_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + +/** + * RRAO command is currently unsupported on this device + */ +int itdtimage_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + int itdtimage_get_eod_status(void *vstate, int partition) { struct itdtimage_data *state = (struct itdtimage_data *)vstate; @@ -1623,6 +1641,8 @@ struct tape_ops itdtimage_handler = { .write_attribute = itdtimage_write_attribute, .read_attribute = itdtimage_read_attribute, .allow_overwrite = itdtimage_allow_overwrite, + .grao = itdtimage_grao, + .rrao = itdtimage_rrao, .set_compression = itdtimage_set_compression, .set_default = itdtimage_set_default, .get_cartridge_health = itdtimage_get_cartridge_health, diff --git a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c index 8985d39d..9eb7496e 100644 --- a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c +++ b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c @@ -2711,6 +2711,24 @@ int lin_tape_ibmtape_allow_overwrite(void *device, const struct tc_position pos) return rc; } +/** + * GRAO command is currently unsupported on this device + */ +int lin_tape_ibmtape_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + +/** + * RRAO command is currently unsupported on this device + */ +int lin_tape_ibmtape_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + /** * Set compression setting * @param device a pointer to the ibmtape backend @@ -4377,6 +4395,8 @@ struct tape_ops lin_tape_ibmtape_drive_handler = { .write_attribute = lin_tape_ibmtape_write_attribute, .read_attribute = lin_tape_ibmtape_read_attribute, .allow_overwrite = lin_tape_ibmtape_allow_overwrite, + .grao = lin_tape_ibmtape_grao, + .rrao = lin_tape_ibmtape_rrao, // May be command combination .set_compression = lin_tape_ibmtape_set_compression, .set_default = lin_tape_ibmtape_set_default, diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index bdab4a52..33d5daf6 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -1169,6 +1169,80 @@ static int _register_key(void *device, unsigned char *key) return ret; } +/** + * Local function to get the uds limits for device when submitting the GRAO command. + * @param device A pointer to the tape device + * @param[out] max_uds_supported max uds supported + * @param[out] max_uds_size max size supported + * @return 0 on success or a negative value on error + */ +static int _get_uds_rao(void *device, uint32_t *max_uds_supported, uint32_t *max_uds_size) +{ + int ret = -EDEV_UNKNOWN; + int ret_ep = DEVICE_GOOD; + + struct sg_data *priv = (struct sg_data*)device; + sg_io_hdr_t req; + unsigned char sense[MAXSENSE]; + unsigned char cdb[CDB12_LEN]; + int timeout; + char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "RRAO"; + char *msg = NULL; + + /* uds mode preset */ + uint32_t len = 4; /* allocation length to be returned */ + uint32_t uds_sa = 0x9D; /* uds limits 1b + service action */ + uint32_t rao_offset = 0x0; /* Rao list offset. Set to zero. */ + + /* Prepare the buffer */ + unsigned char *buffer = calloc(1, len); + if (!buffer) { + ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + return -EDEV_NO_MEMORY; + } + + /* Zero out the CDB and the result buffer */ + ret = init_sg_io_header(&req); + if (ret < 0) + return ret; + memset(cdb, 0, sizeof(cdb)); + memset(sense, 0, sizeof(sense)); + + /* Build CDB */ + cdb[0] = MAINTENANCE_IN; + cdb[1] = uds_sa; + ltfs_u32tobe(cdb + 2, rao_offset); + ltfs_u32tobe(cdb + 6, len); + cdb[10] = LTFS_GEOMETORY_OFF; /* UDS_TYPE */ + + timeout = get_timeout(priv->timeouts, cdb[0]); + if (timeout < 0) + return -EDEV_UNSUPPORETD_COMMAND; + + /* Build request */ + req.dxfer_direction = SCSI_FROM_TARGET_TO_INITIATOR; + req.cmd_len = sizeof(cdb); + req.mx_sb_len = sizeof(sense); + req.dxfer_len = len; + req.dxferp = buffer; + req.cmdp = cdb; + req.sbp = sense; + req.timeout = SGConversion(timeout); + req.usr_ptr = (void *)cmd_desc; + + ret = sg_issue_cdb_command(&priv->dev, &req, &msg); + if (ret == DEVICE_GOOD) { + /* process returned buffer */ + *max_uds_supported = (uint32_t)ltfs_betou16(buffer); + *max_uds_size = (uint32_t)ltfs_betou16(buffer + 2); + } else { + ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); + if (ret_ep < 0) + ret = ret_ep; + } + return ret; +} + /** SCSI command handling of REPORT SUPPORTED OPERATION CODES */ static int _cdb_rsoc(void* device, unsigned char *buf, uint32_t len) @@ -5024,6 +5098,174 @@ int sg_get_block_in_buffer(void *device, uint32_t *block) return ret; } +int sg_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +{ + int ret = -EDEV_UNKNOWN; + int ret_ep = DEVICE_GOOD; + + /* Precheck: Check supported uds size from drive */ + uint32_t max_uds_supported; /* max size of files that can be handled */ + uint32_t max_uds_size; /* max size to be retunred in rrao (32 if LTFS_GEOMETORY_OFF) */ + ret = _get_uds_rao(device, &max_uds_supported, &max_uds_size); + if (ret < 0) { + /* Failed to get supported uds size from drive. */ + ltfsmsg(LTFS_ERR, 17275E); + return -ret; + } + if (num_of_files >= max_uds_supported) { + ltfsmsg(LTFS_ERR, 17276E, max_uds_supported, max_uds_size); + return -EDEV_INVALID_ARG; + } + + /* GRAO starts here */ + struct sg_data *priv = (struct sg_data*)device; + sg_io_hdr_t req; + unsigned char cdb[CDB12_LEN]; + unsigned char sense[MAXSENSE]; + int timeout; + char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "GRAO"; + char *msg = NULL; + + /* Prepare the buffer (Parameter List) to transfer */ + uint32_t len = 32 * num_of_files + 8; /* (uds siz * num of file) + 0-7 byte header */ + unsigned char *buffer = calloc(1, len); + if (!buffer) { + ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + return -EDEV_NO_MEMORY; + } + + /* Zero out the CDB and the result buffer */ + ret = init_sg_io_header(&req); + if (ret < 0) + return ret; + + memset(cdb, 0, sizeof(cdb)); + memset(sense, 0, sizeof(sense)); + + /* Build CDB */ + cdb[0] = MAINTENANCE_OUT; /* op_code */ + cdb[1] = 0x1D; /* service action */ + cdb[2] = 0x2; /* PROCESS, reorder UDS on */ + cdb[3] = LTFS_GEOMETORY_OFF; /* UDS_TYPE */ + ltfs_u32tobe(cdb + 6, len); /* parameter list len starts from 6 byte, adding len to cbc 6-9 */ + + timeout = get_timeout(priv->timeouts, cdb[0]); + if (timeout < 0) + return -EDEV_UNSUPPORETD_COMMAND; + + /* Build request */ + req.dxfer_direction = SCSI_FROM_INITIATOR_TO_TARGET; + req.cmd_len = sizeof(cdb); + req.mx_sb_len = sizeof(sense); + req.dxfer_len = len; + req.dxferp = buffer; + req.cmdp = cdb; + req.sbp = sense; + req.timeout = SGConversion(timeout); + req.usr_ptr = (void *)cmd_desc; + + if (ret < 0){ + ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); + if (ret_ep < 0) + ret = ret_ep; + } + free(buffer); + + return ret; +} + +int sg_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +{ + int ret = -EDEV_UNKNOWN; + int ret_ep = DEVICE_GOOD; + + struct sg_data *priv = (struct sg_data*)device; + sg_io_hdr_t req; + unsigned char sense[MAXSENSE]; + unsigned char cdb[CDB12_LEN]; + int timeout; + char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "RRAO"; + char *msg = NULL; + + /* Allocation length to be returned, initial size enough to get additional data */ + uint64_t len = 8; + + uint32_t uds_sa = 0x1D; /* uds limits + service action */ + uint32_t rao_offset = 0x0; /* Rao list offset. Set to zero. */ + + /* Prepare the buffer */ + unsigned char *tmp_buf = calloc(1, len); + if (!tmp_buf) { + ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + return -EDEV_NO_MEMORY; + } + + /* Zero out the CDB and the result buffer */ + ret = init_sg_io_header(&req); + if (ret < 0) + return ret; + memset(cdb, 0, sizeof(cdb)); + memset(sense, 0, sizeof(sense)); + + /* Build CDB */ + cdb[0] = MAINTENANCE_IN; + cdb[1] = uds_sa; + ltfs_u32tobe(cdb + 2, rao_offset); + ltfs_u32tobe(cdb + 6, len); + cdb[10] = LTFS_GEOMETORY_OFF; /* UDS_TYPE */ + + timeout = get_timeout(priv->timeouts, cdb[0]); + if (timeout < 0) + return -EDEV_UNSUPPORETD_COMMAND; + + /* Build request */ + req.dxfer_direction = SCSI_FROM_TARGET_TO_INITIATOR; + req.cmd_len = sizeof(cdb); + req.mx_sb_len = sizeof(sense); + req.dxfer_len = len; + req.dxferp = tmp_buf; + req.cmdp = cdb; + req.sbp = sense; + req.timeout = SGConversion(timeout); + req.usr_ptr = (void *)cmd_desc; + + ret = sg_issue_cdb_command(&priv->dev, &req, &msg); + if (ret < 0){ + ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); + if (ret_ep < 0) + ret = ret_ep; + free(tmp_buf); + return ret; + } + + /* check tmp_buf size and reset len */ + len = (uint64_t)ltfs_betou32(tmp_buf + 4); + free(tmp_buf); + + /* Re-prepare the buffer and get full data */ + unsigned char *buffer = calloc(1, len); + if (!buffer) { + ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + return -EDEV_NO_MEMORY; + } + req.dxfer_len = len; + req.dxferp = buffer; + + ret = sg_issue_cdb_command(&priv->dev, &req, &msg); + if (ret < 0) { + ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); + if (ret_ep < 0) + ret = ret_ep; + } + + /* copy data */ + memcpy(out_buf, buffer, len); + memcpy(out_size, &len, sizeof(size_t)); + free(buffer); + + return ret; +} + struct tape_ops sg_handler = { .open = sg_open, .reopen = sg_reopen, @@ -5056,6 +5298,8 @@ struct tape_ops sg_handler = { .write_attribute = sg_write_attribute, .read_attribute = sg_read_attribute, .allow_overwrite = sg_allow_overwrite, + .grao = sg_grao, + .rrao = sg_rrao, // May be command combination .set_compression = sg_set_compression, .set_default = sg_set_default, @@ -5100,4 +5344,4 @@ const char *tape_dev_get_message_bundle_name(void **message_data) { *message_data = tape_linux_sg_dat; return "tape_linux_sg"; -} +} \ No newline at end of file diff --git a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c index 18765d1c..21319327 100644 --- a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c +++ b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c @@ -3218,6 +3218,24 @@ int scsipi_ibmtape_allow_overwrite(void *device, const struct tc_position pos) return ret; } +/** + * GRAO command is currently unsupported on this device + */ +int scsipi_ibmtape_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + +/** + * RRAO command is currently unsupported on this device + */ +int scsipi_ibmtape_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + int scsipi_ibmtape_set_compression(void *device, const bool enable_compression, struct tc_position *pos) { int ret = -EDEV_UNKNOWN; @@ -4574,6 +4592,8 @@ struct tape_ops scsipi_ibmtape_handler = { .write_attribute = scsipi_ibmtape_write_attribute, .read_attribute = scsipi_ibmtape_read_attribute, .allow_overwrite = scsipi_ibmtape_allow_overwrite, + .grao = scsipi_ibmtape_grao, + .rrao = scsipi_ibmtape_rrao, // May be command combination .set_compression = scsipi_ibmtape_set_compression, .set_default = scsipi_ibmtape_set_default, diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index ef25be56..fd60e36f 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -2987,6 +2987,24 @@ int iokit_allow_overwrite(void *device, const struct tc_position pos) return ret; } +/** + * GRAO command is currently unsupported on this device + */ +int iokit_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + +/** + * RRAO command is currently unsupported on this device + */ +int iokit_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +{ + int ret = -EDEV_UNSUPPORETD_COMMAND; + return ret; +} + int iokit_set_compression(void *device, const bool enable_compression, struct tc_position *pos) { int ret = -EDEV_UNKNOWN; @@ -4304,6 +4322,8 @@ struct tape_ops iokit_handler = { .write_attribute = iokit_write_attribute, .read_attribute = iokit_read_attribute, .allow_overwrite = iokit_allow_overwrite, + .grao = iokit_grao, + .rrao = iokit_rrao, // May be command combination .set_compression = iokit_set_compression, .set_default = iokit_set_default, From 54a0c5cc0db1b1ce933e08a3700c63b167fc86bf Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Thu, 10 Jun 2021 18:19:24 +0900 Subject: [PATCH 037/121] Refactor RAO functions (#293) --- messages/libltfs/root.txt | 16 +- src/libltfs/ltfs.c | 184 +++++++++--------- src/libltfs/ltfs.h | 2 +- src/libltfs/tape.c | 35 +--- src/libltfs/tape.h | 11 +- src/libltfs/tape_ops.h | 12 +- src/libltfs/xml_reader_libltfs.c | 2 +- src/tape_drivers/freebsd/cam/cam_tc.c | 4 +- src/tape_drivers/generic/file/filedebug_tc.c | 4 +- src/tape_drivers/generic/itdtimg/itdtimg_tc.c | 4 +- .../linux/lin_tape/lin_tape_ibmtape.c | 4 +- src/tape_drivers/linux/sg/sg_tape.c | 176 +++-------------- .../netbsd/scsipi-ibmtape/scsipi_ibmtape.c | 4 +- src/tape_drivers/osx/iokit/iokit_tape.c | 4 +- 14 files changed, 171 insertions(+), 291 deletions(-) diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index e177b796..4e0afe47 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -816,14 +816,14 @@ v 17272E:string { "XML parser: Detected invalid %s in a directory (%s)." } 17273E:string { "XML parser: Detected an I/O error on reading an index from the tape (%d)." } 17274W:string { "XML parser: Unexpected error code is detected (%d)." } - 17275E:string { "Failed to get supported uds size from drive." } - 17276E:string { "Too many files to process on this drive. Valid size: max_uds_supported: %u, max_uds_size: %u." } - 17277D:string { "RAO list clear: %s." } - 17278E:string { "RAO command returns error on %s (%d)." } - 17279E:string { "File number exceeds the maximum limit for RAO. Maximum file supported: %u, given: %u." } - 17280E:string { "Error occured when reading file set for RAO: %s." } - 17281E:string { "Failed to create RAO output file: %s." } - 17282E:string { "Failed to read RAO input file: %s." } + 17275I:string { "RAO command returns error on %s (%d)." } + 17276I:string { "Failed to create RAO output file: %s (%d)." } + 17277I:string { "Failed to write to RAO output file: %s (%d)." } + 17278I:string { "Written length to RAO output file is unexpected: %s (Actual %ld, Expected %ld)." } + 17279I:string { "Failed to open RAO input file: %s (%d)." } + 17280I:string { "Failed to get the length of the RAO input file: %s (%d)." } + 17281I:string { "Failed to read from RAO input file: %s (%d)." } + 17282I:string { "Read length from RAO input file is unexpected: %s (Actual %ld, Expected %ld)." } // For Debug 19999I:string { "%s %s %d." } diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index e6442150..5c865845 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -81,6 +81,11 @@ #define MIN(a, b) ((a) < (b) ? (a) : (b)) #endif +/* O_BINARY is defined only in MinGW */ +#ifndef O_BINARY +#define O_BINARY 0 +#endif + volatile char *copyright = LTFS_COPYRIGHT_0"\n"LTFS_COPYRIGHT_1"\n"LTFS_COPYRIGHT_2"\n" \ LTFS_COPYRIGHT_3"\n"LTFS_COPYRIGHT_4"\n"LTFS_COPYRIGHT_5"\n"; @@ -4228,89 +4233,98 @@ int ltfs_profiler_set(uint64_t source, struct ltfs_volume *vol) return ret; } -static int _ltfs_write_rao_file(char *write_data, char *file_path, size_t *write_size) +static int _ltfs_write_rao_file(char *file_path_org, unsigned char *buf, size_t len) { - int rc = -EDEV_UNKNOWN; + int ret = -EDEV_UNKNOWN; char *path; + int fd = -1; + ssize_t size = 0; - rc = asprintf(&path, "%s%s", file_path, LTFS_OUT_FILE_EXTENSION); - if (rc < 0) { + ret = asprintf(&path, "%s%s", file_path_org, LTFS_OUT_FILE_EXTENSION); + if (ret < 0) { ltfsmsg(LTFS_ERR, 10001E, __FILE__); return -LTFS_NO_MEMORY; } - FILE *p; - p = fopen(path, "wb"); - if (!p) { - ltfsmsg(LTFS_ERR, 17281E, "File open failed"); - return -LTFS_FILE_ERR; + fd = open(path, + O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, + S_IWUSR | S_IRUSR | S_IRGRP | S_IROTH); + if (fd < 0) { + ltfsmsg(LTFS_INFO, 17276I, path, errno); + free(path); + ret = -errno; + return ret; + } + + size = write(fd, buf, len); + if (size < 0) { + ltfsmsg(LTFS_INFO, 17277I, path, errno); + ret = -errno; + goto out; + } else if (size != (ssize_t)len) { + ltfsmsg(LTFS_INFO, 17278I, path, size, (ssize_t)len); + ret = LTFS_FILE_ERR; + goto out; } else { - fwrite(&write_data, sizeof(char), (size_t)write_size, p); - fclose(p); - rc = DEVICE_GOOD; + ret = 0; } - return rc; + fsync(fd); /* Make sure the data is written to the device */ + +out: + free(path); + close(fd); + return ret; } -static int _ltfs_read_rao_file(char *read_data, uint32_t *num_of_files, char *file_path) +static int _ltfs_read_rao_file(char *file_path, unsigned char *buf, + size_t len, uint32_t *in_size) { - int rc = -EDEV_UNKNOWN; + int ret = -EDEV_UNKNOWN; char *path; - int size = 0; + int fd = -1; + struct stat sbuf; + ssize_t size = 0; - rc = asprintf(&path, "%s", file_path); - if (rc < 0) { + ret = asprintf(&path, "%s", file_path); + if (ret < 0) { ltfsmsg(LTFS_ERR, 10001E, __FILE__); return -LTFS_NO_MEMORY; } - /* file reader */ - FILE *p; - p = fopen(path, "rb"); - if (!p) { - ltfsmsg(LTFS_ERR, 17282E, "File open failed"); - return -EDEV_INVALID_ARG; - } - - rc = fseeko(p, 0LL, SEEK_END); - if(rc < 0){ - ltfsmsg(LTFS_ERR, 17282E, "File seek failed"); - return -LTFS_FILE_ERR; - } - size = ftello(p); - - if (size >= RAO_MAX_RET_SIZE) { - ltfsmsg(LTFS_ERR, 17282E, "File size is too big"); - return -EDEV_INVALID_ARG; - } else if (size < 0) { - ltfsmsg(LTFS_ERR, 17282E, "File seek failed"); - return -LTFS_FILE_ERR; - } - - rc = fseeko(p,0,SEEK_SET); - if (rc < 0) { - ltfsmsg(LTFS_ERR, 17282E, "File seek failed"); - return -LTFS_FILE_ERR; + fd = open(path, O_RDONLY | O_BINARY); + if (fd < 0) { + ltfsmsg(LTFS_INFO, 17279I, path, errno); + free(path); + ret = -errno; + return ret; } - rc = fread(read_data, sizeof(char), size, p); - fclose(p); - if (rc < 0) { - ltfsmsg(LTFS_ERR, 17282E, "File read failed"); - return -LTFS_FILE_ERR; + ret = fstat(fd, &sbuf); + if (ret < 0) { + ret = -errno; + ltfsmsg(LTFS_INFO, 17280I, path, errno); + goto out; } - /* Input parser, counts how many files (num_of_files) there are in the input. */ - /* The grao param list size should be Header(8 bytes) + ( UDS(32 bytes) * num_of_files ). */ - if ( (size - 8) % 32 != 0 || size < 8+32 ) { - /* data format is wrong */ - ltfsmsg(LTFS_ERR, 17282E, "Unreadable file format"); - return -EDEV_INVALID_ARG; + size = read(fd, buf, len); + if (size < 0) { + ltfsmsg(LTFS_INFO, 17281I, path, errno); + ret = -errno; + goto out; + } if (size != (ssize_t)sbuf.st_size) { + ltfsmsg(LTFS_INFO, 17282I, path, size, (ssize_t)sbuf.st_size); + ret = LTFS_FILE_ERR; + goto out; + } else { + ret = 0; + *in_size = (uint32_t)size; } - *num_of_files = (size - 8) / 32; - return rc; +out: + free(path); + close(fd); + return ret; } /** @@ -4322,53 +4336,49 @@ static int _ltfs_read_rao_file(char *read_data, uint32_t *num_of_files, char *fi int ltfs_get_rao_list(char *path, struct ltfs_volume *vol) { int ret = -EDEV_UNKNOWN; + struct rao_mod rao; + + CHECK_ARG_NULL(path, -LTFS_NULL_ARG); + CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); + + memset(&rao, 0, sizeof(struct rao_mod)); + rao.in_buf = calloc(1, RAO_MAX_RET_SIZE); + if (!rao.in_buf) { + ltfsmsg(LTFS_ERR, 10001E, "ltfs_get_rao_list: out_buf"); + return -ENOMEM; + } + + rao.out_buf = calloc(1, RAO_MAX_RET_SIZE); + if (!rao.out_buf) { + ltfsmsg(LTFS_ERR, 10001E, "ltfs_get_rao_list: out_buf"); + free(rao.in_buf); + return -ENOMEM; + } + + rao.buf_size = RAO_MAX_RET_SIZE; + ret = tape_device_lock(vol->device); if (ret < 0) { ltfsmsg(LTFS_ERR, 12010E, __FUNCTION__); return ret; } - /* Prepare memory and read data from file */ - struct rao_mod rao; - vol->device->rao = &rao; - char in_buf[RAO_MAX_RET_SIZE]; /* read bytes is never larger than return size */ - rao.in_buf = (char *)&in_buf; - - if (path == NULL){ - ret = -EDEV_INVALID_ARG; - goto out; - } - /* get rao */ - uint32_t *tmp_num = &rao.num_of_files; - ret = _ltfs_read_rao_file(rao.in_buf, tmp_num, path); + ret = _ltfs_read_rao_file(path, rao.in_buf, RAO_MAX_RET_SIZE, &rao.in_size); if (ret < 0) { goto out; } - /* file size check */ - if (rao.num_of_files >= RAO_MAX_FILENUM) { - ltfsmsg(LTFS_ERR, 17279E, RAO_MAX_FILENUM, rao.num_of_files); - ret = -EDEV_INVALID_ARG; - goto out; - } - /* Send RAO request*/ - ret = tape_rao_request(vol->device, vol->device->rao); + ret = tape_rao_request(vol->device, &rao); if (ret < 0) { goto out; } /* write data to file */ - ret = _ltfs_write_rao_file(rao.out_buf, path, rao.out_size); - if (ret < 0) { - goto out; - } - - goto out; + ret = _ltfs_write_rao_file(path, rao.out_buf, rao.out_size); out: tape_device_unlock(vol->device); - vol->device->rao = NULL; return ret; -} \ No newline at end of file +} diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index d2eb61b2..34aabe40 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -174,7 +174,7 @@ struct device_data; #define LTFS_GEOMETORY_OFF (0x00) #define LTFS_GEOMETORY_ON (0x01) /* Maximum size allowed that is returned from RAO */ -#define RAO_MAX_RET_SIZE RAO_MAX_FILENUM * (32 + LTFS_GEOMETORY_OFF*20) + 8 +#define RAO_MAX_RET_SIZE ((RAO_MAX_FILENUM * (32 + (LTFS_GEOMETORY_OFF * 20))) + 8) #define INTERRUPTED_GOTO(rc, label) \ do{ \ diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index cffaaebe..892f1d6a 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -721,11 +721,12 @@ int tape_test_unit_ready(struct device_data *dev) } ret = _tape_test_unit_ready(dev); - if (ret < 0) + if (ret < 0) { ltfsmsg(LTFS_ERR, 12029E, ret); - - dev->previous_exist.tv_sec = ts_now.tv_sec; - dev->previous_exist.tv_nsec = ts_now.tv_nsec; + } else { + dev->previous_exist.tv_sec = ts_now.tv_sec; + dev->previous_exist.tv_nsec = ts_now.tv_nsec; + } return ret; } @@ -3576,41 +3577,25 @@ int tape_set_profiler(struct device_data *dev, char *work_dir, bool enable) * All data should be set in rao before this is called. * @param dev a pointer to the tape device * @param rao a pointer to the rao data - * @param [out] ret_buf the returned buffer from grao. * @return 0 on success or a negative value on error */ int tape_rao_request(struct device_data *dev, struct rao_mod *rao) { int ret = 0; - /* check file size */ - if (rao->num_of_files <= 0) { - /* rao list will be cleared if call size is less than zero */ - rao->in_buf = NULL; - ltfsmsg(LTFS_DEBUG, 17277D, "Clear Called"); - } - - if (rao->in_buf == NULL){ - return EDEV_INTERNAL_ERROR; - } - /* run GRAO */ - ret = dev->backend->grao(dev->backend_data, (const unsigned char *)rao->in_buf, rao->num_of_files); + ret = dev->backend->grao(dev->backend_data, rao->in_buf, rao->in_size); if (ret < 0) { - ltfsmsg(LTFS_ERR, 17278E, "GRAO", ret); //GRAO command returns error - return ret; - } else if (rao->num_of_files <= 0) { - /* rao list cleared */ - ltfsmsg(LTFS_DEBUG, 17277D, "Clear Done"); + ltfsmsg(LTFS_INFO, 17275I, "GRAO", ret); //GRAO command returns error return ret; } /* run RRAO */ - ret = dev->backend->rrao(dev->backend_data, rao->num_of_files, (char *)&rao->out_buf, (size_t *)&rao->out_size); + ret = dev->backend->rrao(dev->backend_data, rao->out_buf, rao->buf_size, &rao->out_size); if (ret < 0) { - ltfsmsg(LTFS_ERR, 17278E, "RRAO", ret); + ltfsmsg(LTFS_INFO, 17275I, "RRAO", ret); return ret; } return ret; -} \ No newline at end of file +} diff --git a/src/libltfs/tape.h b/src/libltfs/tape.h index 6020ff8f..0901c091 100644 --- a/src/libltfs/tape.h +++ b/src/libltfs/tape.h @@ -87,10 +87,11 @@ extern "C" { #define IS_UNEXPECTED_MOVE(ret) (ret == -EDEV_MEDIUM_REMOVAL_REQ) struct rao_mod { - uint32_t num_of_files; /* number of files to process */ - char *in_buf; /* buffer to set in grao */ - char *out_buf; /* buffer returned from rrao */ - size_t *out_size; /* buffer size returned in out_buf */ + unsigned char *in_buf; /**< buffer to set in grao */ + unsigned char *out_buf; /**< buffer returned from rrao */ + size_t buf_size; /**< buffer size of in_buf and out_buf */ + uint32_t in_size; /**< Valid length of in_buf */ + size_t out_size; /**< Valid length of out_buf */ }; struct device_data { @@ -115,8 +116,6 @@ struct device_data { ltfs_mutex_t backend_mutex; /**< Mutex to control backend access */ ltfs_mutex_t read_only_flag_mutex; /**< Mutex to control read_only access */ char *serial_number; /**< Serial number for identification */ - - struct rao_mod *rao; /**< RAO related module */ }; int tape_device_alloc(struct device_data **device); diff --git a/src/libltfs/tape_ops.h b/src/libltfs/tape_ops.h index 6a1af0e6..f4a51a8c 100644 --- a/src/libltfs/tape_ops.h +++ b/src/libltfs/tape_ops.h @@ -933,21 +933,21 @@ struct tape_ops { * The generated result can be called from the RRAO command. * @param device A pointer to the tape device * @param buf buffer data to process - * @param num_of_files The number of files to process for RAO. + * @param len Length of the buffer * @return 0 on success or a negative value on error */ - int (*grao)(void *device, const unsigned char *buf, const uint32_t num_of_files); + int (*grao)(void *device, unsigned char *buf, const uint32_t len); /** * Submit the RRAO (Receive Recommended Access Order) command. * This should be called after the GRAO command. * @param device A pointer to the tape device - * @param[out] out_buf rrao, rao result in binary. You need to parse it by yourself. - * @param[out] out_size size of out_buf that is retuned. + * @param[out] buf rrao, rao result in binary. You need to parse it by yourself. + * @param len Length of the buffer + * @param[out] out_size size of valid data returned from the device * @return 0 on success or a negative value on error */ - int (*rrao)(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size); - + int (*rrao)(void *device, unsigned char *buf, const uint32_t len, size_t *out_size); }; /** diff --git a/src/libltfs/xml_reader_libltfs.c b/src/libltfs/xml_reader_libltfs.c index 1a3b2be0..92ba852b 100644 --- a/src/libltfs/xml_reader_libltfs.c +++ b/src/libltfs/xml_reader_libltfs.c @@ -1960,7 +1960,7 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) ltfsmsg(LTFS_ERR, 17016E, ret); if (ret == -1) { /* Unexpected error code, we need to investigate */ - ltfsmsg(LTFS_ERR, 17274W, ret); + ltfsmsg(LTFS_WARN, 17274W, ret); ret = -LTFS_INDEX_INVALID; } } else if (ret == 0) { diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 856bf094..63a85ebe 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -2314,7 +2314,7 @@ int camtape_allow_overwrite(void *device, const struct tc_position pos) /** * GRAO command is currently unsupported on this device */ -int camtape_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +int camtape_grao(void *device, unsigned char *buf, const uint32_t len) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; @@ -2323,7 +2323,7 @@ int camtape_grao(void *device, const unsigned char *buf, const uint32_t num_of_f /** * RRAO command is currently unsupported on this device */ -int camtape_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +int camtape_rrao(void *device, unsigned char *buf, const uint32_t len, size_t *out_size) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; diff --git a/src/tape_drivers/generic/file/filedebug_tc.c b/src/tape_drivers/generic/file/filedebug_tc.c index fa934d1f..b9f6835b 100644 --- a/src/tape_drivers/generic/file/filedebug_tc.c +++ b/src/tape_drivers/generic/file/filedebug_tc.c @@ -2114,7 +2114,7 @@ int filedebug_allow_overwrite(void *device, const struct tc_position pos) /** * GRAO command is currently unsupported on this device */ -int filedebug_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +int filedebug_grao(void *device, unsigned char *buf, const uint32_t len) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; @@ -2123,7 +2123,7 @@ int filedebug_grao(void *device, const unsigned char *buf, const uint32_t num_of /** * RRAO command is currently unsupported on this device */ -int filedebug_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +int filedebug_rrao(void *device, unsigned char *buf, const uint32_t len, size_t *out_size) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; diff --git a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c index 5809dd47..ec00d949 100644 --- a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c +++ b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c @@ -965,7 +965,7 @@ int itdtimage_allow_overwrite(void *device, const struct tc_position pos) /** * GRAO command is currently unsupported on this device */ -int itdtimage_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +int itdtimage_grao(void *device, unsigned char *buf, const uint32_t len) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; @@ -974,7 +974,7 @@ int itdtimage_grao(void *device, const unsigned char *buf, const uint32_t num_of /** * RRAO command is currently unsupported on this device */ -int itdtimage_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +int itdtimage_rrao(void *device, unsigned char *buf, const uint32_t len, size_t *out_size) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; diff --git a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c index 9eb7496e..b9efb827 100644 --- a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c +++ b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c @@ -2714,7 +2714,7 @@ int lin_tape_ibmtape_allow_overwrite(void *device, const struct tc_position pos) /** * GRAO command is currently unsupported on this device */ -int lin_tape_ibmtape_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +int lin_tape_ibmtape_grao(void *device, unsigned char *buf, const uint32_t len) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; @@ -2723,7 +2723,7 @@ int lin_tape_ibmtape_grao(void *device, const unsigned char *buf, const uint32_t /** * RRAO command is currently unsupported on this device */ -int lin_tape_ibmtape_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +int lin_tape_ibmtape_rrao(void *device, unsigned char *buf, const uint32_t len, size_t *out_size) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index 33d5daf6..f5830792 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -1169,80 +1169,6 @@ static int _register_key(void *device, unsigned char *key) return ret; } -/** - * Local function to get the uds limits for device when submitting the GRAO command. - * @param device A pointer to the tape device - * @param[out] max_uds_supported max uds supported - * @param[out] max_uds_size max size supported - * @return 0 on success or a negative value on error - */ -static int _get_uds_rao(void *device, uint32_t *max_uds_supported, uint32_t *max_uds_size) -{ - int ret = -EDEV_UNKNOWN; - int ret_ep = DEVICE_GOOD; - - struct sg_data *priv = (struct sg_data*)device; - sg_io_hdr_t req; - unsigned char sense[MAXSENSE]; - unsigned char cdb[CDB12_LEN]; - int timeout; - char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "RRAO"; - char *msg = NULL; - - /* uds mode preset */ - uint32_t len = 4; /* allocation length to be returned */ - uint32_t uds_sa = 0x9D; /* uds limits 1b + service action */ - uint32_t rao_offset = 0x0; /* Rao list offset. Set to zero. */ - - /* Prepare the buffer */ - unsigned char *buffer = calloc(1, len); - if (!buffer) { - ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); - return -EDEV_NO_MEMORY; - } - - /* Zero out the CDB and the result buffer */ - ret = init_sg_io_header(&req); - if (ret < 0) - return ret; - memset(cdb, 0, sizeof(cdb)); - memset(sense, 0, sizeof(sense)); - - /* Build CDB */ - cdb[0] = MAINTENANCE_IN; - cdb[1] = uds_sa; - ltfs_u32tobe(cdb + 2, rao_offset); - ltfs_u32tobe(cdb + 6, len); - cdb[10] = LTFS_GEOMETORY_OFF; /* UDS_TYPE */ - - timeout = get_timeout(priv->timeouts, cdb[0]); - if (timeout < 0) - return -EDEV_UNSUPPORETD_COMMAND; - - /* Build request */ - req.dxfer_direction = SCSI_FROM_TARGET_TO_INITIATOR; - req.cmd_len = sizeof(cdb); - req.mx_sb_len = sizeof(sense); - req.dxfer_len = len; - req.dxferp = buffer; - req.cmdp = cdb; - req.sbp = sense; - req.timeout = SGConversion(timeout); - req.usr_ptr = (void *)cmd_desc; - - ret = sg_issue_cdb_command(&priv->dev, &req, &msg); - if (ret == DEVICE_GOOD) { - /* process returned buffer */ - *max_uds_supported = (uint32_t)ltfs_betou16(buffer); - *max_uds_size = (uint32_t)ltfs_betou16(buffer + 2); - } else { - ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); - if (ret_ep < 0) - ret = ret_ep; - } - return ret; -} - /** SCSI command handling of REPORT SUPPORTED OPERATION CODES */ static int _cdb_rsoc(void* device, unsigned char *buf, uint32_t len) @@ -5098,26 +5024,11 @@ int sg_get_block_in_buffer(void *device, uint32_t *block) return ret; } -int sg_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +int sg_grao(void *device, unsigned char *buf, const uint32_t len) { int ret = -EDEV_UNKNOWN; int ret_ep = DEVICE_GOOD; - /* Precheck: Check supported uds size from drive */ - uint32_t max_uds_supported; /* max size of files that can be handled */ - uint32_t max_uds_size; /* max size to be retunred in rrao (32 if LTFS_GEOMETORY_OFF) */ - ret = _get_uds_rao(device, &max_uds_supported, &max_uds_size); - if (ret < 0) { - /* Failed to get supported uds size from drive. */ - ltfsmsg(LTFS_ERR, 17275E); - return -ret; - } - if (num_of_files >= max_uds_supported) { - ltfsmsg(LTFS_ERR, 17276E, max_uds_supported, max_uds_size); - return -EDEV_INVALID_ARG; - } - - /* GRAO starts here */ struct sg_data *priv = (struct sg_data*)device; sg_io_hdr_t req; unsigned char cdb[CDB12_LEN]; @@ -5126,12 +5037,12 @@ int sg_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "GRAO"; char *msg = NULL; - /* Prepare the buffer (Parameter List) to transfer */ - uint32_t len = 32 * num_of_files + 8; /* (uds siz * num of file) + 0-7 byte header */ - unsigned char *buffer = calloc(1, len); - if (!buffer) { - ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); - return -EDEV_NO_MEMORY; + if (IS_LTO(priv->drive_type)) { + if ( DRIVE_GEN(priv->drive_type) == 0x05 || DRIVE_GEN(priv->drive_type) == 0x06 || + DRIVE_GEN(priv->drive_type) == 0x07 || DRIVE_GEN(priv->drive_type) == 0x08 ) { + /* LTO6-LTO8 don't support RAO commands */ + return -EDEV_UNSUPPORETD_COMMAND; + } } /* Zero out the CDB and the result buffer */ @@ -5143,11 +5054,11 @@ int sg_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) memset(sense, 0, sizeof(sense)); /* Build CDB */ - cdb[0] = MAINTENANCE_OUT; /* op_code */ - cdb[1] = 0x1D; /* service action */ - cdb[2] = 0x2; /* PROCESS, reorder UDS on */ - cdb[3] = LTFS_GEOMETORY_OFF; /* UDS_TYPE */ - ltfs_u32tobe(cdb + 6, len); /* parameter list len starts from 6 byte, adding len to cbc 6-9 */ + cdb[0] = MAINTENANCE_OUT; /* op_code */ + cdb[1] = 0x1D; /* service action */ + cdb[2] = 0x2; /* PROCESS, reorder UDS on */ + cdb[3] = LTFS_GEOMETORY_OFF; /* UDS_TYPE */ + ltfs_u32tobe(cdb + 6, len); /* parameter list len starts from 6 byte, adding len to cbc 6-9 */ timeout = get_timeout(priv->timeouts, cdb[0]); if (timeout < 0) @@ -5158,23 +5069,23 @@ int sg_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) req.cmd_len = sizeof(cdb); req.mx_sb_len = sizeof(sense); req.dxfer_len = len; - req.dxferp = buffer; + req.dxferp = buf; req.cmdp = cdb; req.sbp = sense; req.timeout = SGConversion(timeout); req.usr_ptr = (void *)cmd_desc; + ret = sg_issue_cdb_command(&priv->dev, &req, &msg); if (ret < 0){ ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); if (ret_ep < 0) ret = ret_ep; } - free(buffer); return ret; } -int sg_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +int sg_rrao(void *device, unsigned char *buf, const uint32_t len, size_t *out_size) { int ret = -EDEV_UNKNOWN; int ret_ep = DEVICE_GOOD; @@ -5187,17 +5098,14 @@ int sg_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *ou char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "RRAO"; char *msg = NULL; - /* Allocation length to be returned, initial size enough to get additional data */ - uint64_t len = 8; - - uint32_t uds_sa = 0x1D; /* uds limits + service action */ - uint32_t rao_offset = 0x0; /* Rao list offset. Set to zero. */ + *out_size = 0; - /* Prepare the buffer */ - unsigned char *tmp_buf = calloc(1, len); - if (!tmp_buf) { - ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); - return -EDEV_NO_MEMORY; + if (IS_LTO(priv->drive_type)) { + if ( DRIVE_GEN(priv->drive_type) == 0x05 || DRIVE_GEN(priv->drive_type) == 0x06 || + DRIVE_GEN(priv->drive_type) == 0x07 || DRIVE_GEN(priv->drive_type) == 0x08 ) { + /* LTO6-LTO8 don't support RAO commands */ + return -EDEV_UNSUPPORETD_COMMAND; + } } /* Zero out the CDB and the result buffer */ @@ -5209,48 +5117,26 @@ int sg_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *ou /* Build CDB */ cdb[0] = MAINTENANCE_IN; - cdb[1] = uds_sa; - ltfs_u32tobe(cdb + 2, rao_offset); + cdb[1] = 0x1D; /* Do not set UDS_LIMITS (0x80) */ ltfs_u32tobe(cdb + 6, len); cdb[10] = LTFS_GEOMETORY_OFF; /* UDS_TYPE */ timeout = get_timeout(priv->timeouts, cdb[0]); - if (timeout < 0) + if (timeout < 0) { return -EDEV_UNSUPPORETD_COMMAND; + } /* Build request */ req.dxfer_direction = SCSI_FROM_TARGET_TO_INITIATOR; req.cmd_len = sizeof(cdb); req.mx_sb_len = sizeof(sense); req.dxfer_len = len; - req.dxferp = tmp_buf; + req.dxferp = buf; req.cmdp = cdb; req.sbp = sense; req.timeout = SGConversion(timeout); req.usr_ptr = (void *)cmd_desc; - ret = sg_issue_cdb_command(&priv->dev, &req, &msg); - if (ret < 0){ - ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); - if (ret_ep < 0) - ret = ret_ep; - free(tmp_buf); - return ret; - } - - /* check tmp_buf size and reset len */ - len = (uint64_t)ltfs_betou32(tmp_buf + 4); - free(tmp_buf); - - /* Re-prepare the buffer and get full data */ - unsigned char *buffer = calloc(1, len); - if (!buffer) { - ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); - return -EDEV_NO_MEMORY; - } - req.dxfer_len = len; - req.dxferp = buffer; - ret = sg_issue_cdb_command(&priv->dev, &req, &msg); if (ret < 0) { ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); @@ -5258,10 +5144,10 @@ int sg_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *ou ret = ret_ep; } - /* copy data */ - memcpy(out_buf, buffer, len); - memcpy(out_size, &len, sizeof(size_t)); - free(buffer); + if (!ret) { + /* Set real length returned from the drive */ + *out_size = ltfs_betou32(buf + 4) + 8; /* Additional Data length and header */ + } return ret; } @@ -5344,4 +5230,4 @@ const char *tape_dev_get_message_bundle_name(void **message_data) { *message_data = tape_linux_sg_dat; return "tape_linux_sg"; -} \ No newline at end of file +} diff --git a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c index 21319327..38e50173 100644 --- a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c +++ b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c @@ -3221,7 +3221,7 @@ int scsipi_ibmtape_allow_overwrite(void *device, const struct tc_position pos) /** * GRAO command is currently unsupported on this device */ -int scsipi_ibmtape_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +int scsipi_ibmtape_grao(void *device, unsigned char *buf, const uint32_t len) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; @@ -3230,7 +3230,7 @@ int scsipi_ibmtape_grao(void *device, const unsigned char *buf, const uint32_t n /** * RRAO command is currently unsupported on this device */ -int scsipi_ibmtape_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +int scsipi_ibmtape_rrao(void *device, unsigned char *buf, const uint32_t len, size_t *out_size) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index fd60e36f..379842b9 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -2990,7 +2990,7 @@ int iokit_allow_overwrite(void *device, const struct tc_position pos) /** * GRAO command is currently unsupported on this device */ -int iokit_grao(void *device, const unsigned char *buf, const uint32_t num_of_files) +int iokit_grao(void *device, unsigned char *buf, const uint32_t len) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; @@ -2999,7 +2999,7 @@ int iokit_grao(void *device, const unsigned char *buf, const uint32_t num_of_fil /** * RRAO command is currently unsupported on this device */ -int iokit_rrao(void *device, const uint32_t num_of_files, char *out_buf, size_t *out_size) +int iokit_rrao(void *device, unsigned char *buf, const uint32_t len, size_t *out_size) { int ret = -EDEV_UNSUPPORETD_COMMAND; return ret; From 4eafde61cc36dd636cd5185245fc11bfdfad552a Mon Sep 17 00:00:00 2001 From: piste2750 <32239919+piste2750@users.noreply.github.com> Date: Mon, 21 Jun 2021 10:23:51 +0900 Subject: [PATCH 038/121] Handle Quantum type-B drive (#296) --- src/tape_drivers/freebsd/cam/cam_tc.c | 11 ++- src/tape_drivers/hp_tape.c | 8 +-- src/tape_drivers/ibm_tape.c | 94 ++++++++++++------------- src/tape_drivers/linux/sg/sg_tape.c | 5 +- src/tape_drivers/osx/iokit/iokit_tape.c | 7 +- src/tape_drivers/quantum_tape.c | 12 ++-- src/tape_drivers/tape_drivers.h | 6 +- src/tape_drivers/vendor_compat.c | 14 ---- src/tape_drivers/vendor_compat.h | 1 - 9 files changed, 72 insertions(+), 86 deletions(-) diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 63a85ebe..3e2a4ded 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -279,13 +279,11 @@ int camtape_open(const char *devname, void **handle) ltfsmsg(LTFS_INFO, 31229I, vendor); /* Check the drive is supportable */ - softc->vendor = get_vendor_id(vendor); struct supported_device **cur = get_supported_devs(softc->vendor); while(*cur) { - if ((! strncmp((char*)softc->cd->inq_data.vendor, (*cur)->vendor_id, - strlen((*cur)->vendor_id)) ) && - (! strncmp((char*)softc->cd->inq_data.product, (*cur)->product_id, - strlen((*cur)->product_id)) ) ) { + if ((! strncmp((char*)softc->cd->inq_data.vendor, (*cur)->vendor_id, strlen((*cur)->vendor_id)) ) && + (! strncmp((char*)softc->cd->inq_data.product, (*cur)->product_id, strlen((*cur)->product_id)) ) ) { + priv->vendor = (*cur)->vendor_type; drive_type = (*cur)->drive_type; break; } @@ -1786,7 +1784,8 @@ int camtape_remaining_capacity(void *device, struct tc_remaining_cap *cap) ltfs_profiler_add_entry(softc->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_REMAINCAP)); if ((IS_LTO(softc->drive_type) && (DRIVE_GEN(softc->drive_type) == 0x05)) || - (softc->vendor == VENDOR_HP && IS_LTO(softc->drive_type) && (DRIVE_GEN(softc->drive_type) == 0x06))) { + (softc->vendor == VENDOR_HP && IS_LTO(softc->drive_type) && (DRIVE_GEN(softc->drive_type) == 0x06)) || + (softc->vendor == VENDOR_QUANTUM_B && IS_LTO(softc->drive_type))) { /* Issue LogPage 0x31 */ rc = camtape_logsense(device, LOG_TAPECAPACITY, (uint8_t)0, logdata, LOGSENSEPAGE); diff --git a/src/tape_drivers/hp_tape.c b/src/tape_drivers/hp_tape.c index ad9fd496..b812ab69 100644 --- a/src/tape_drivers/hp_tape.c +++ b/src/tape_drivers/hp_tape.c @@ -63,10 +63,10 @@ #include "libltfs/ltfs_endian.h" struct supported_device *hp_supported_drives[] = { - TAPEDRIVE( HP_VENDOR_ID, "Ultrium 5-SCSI", DRIVE_LTO5, "[Ultrium 5-SCSI]" ), /* HP Ultrium Gen 5 */ - TAPEDRIVE( HP_VENDOR_ID, "Ultrium 6-SCSI", DRIVE_LTO6, "[Ultrium 6-SCSI]" ), /* HP Ultrium Gen 6 */ - TAPEDRIVE( HP_VENDOR_ID, "Ultrium 7-SCSI", DRIVE_LTO7, "[Ultrium 7-SCSI]" ), /* HP Ultrium Gen 7 */ - TAPEDRIVE( HPE_VENDOR_ID, "Ultrium 8-SCSI", DRIVE_LTO8, "[Ultrium 8-SCSI]" ), /* HPE Ultrium Gen 8 */ + TAPEDRIVE( HP_VENDOR_ID, "Ultrium 5-SCSI", VENDOR_HP, DRIVE_LTO5, "[Ultrium 5-SCSI]" ), /* HP Ultrium Gen 5 */ + TAPEDRIVE( HP_VENDOR_ID, "Ultrium 6-SCSI", VENDOR_HP, DRIVE_LTO6, "[Ultrium 6-SCSI]" ), /* HP Ultrium Gen 6 */ + TAPEDRIVE( HP_VENDOR_ID, "Ultrium 7-SCSI", VENDOR_HP, DRIVE_LTO7, "[Ultrium 7-SCSI]" ), /* HP Ultrium Gen 7 */ + TAPEDRIVE( HPE_VENDOR_ID, "Ultrium 8-SCSI", VENDOR_HP, DRIVE_LTO8, "[Ultrium 8-SCSI]" ), /* HPE Ultrium Gen 8 */ /* End of supported_devices */ NULL }; diff --git a/src/tape_drivers/ibm_tape.c b/src/tape_drivers/ibm_tape.c index fe7da9c6..70e5c3c8 100644 --- a/src/tape_drivers/ibm_tape.c +++ b/src/tape_drivers/ibm_tape.c @@ -283,58 +283,58 @@ int num_supported_cart = sizeof(supported_cart)/sizeof(supported_ca int num_supported_density = sizeof(supported_density)/sizeof(supported_density[0]); struct supported_device *ibm_supported_drives[] = { - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD5", DRIVE_LTO5, "[ULTRIUM-TD5]" ), /* IBM Ultrium Gen 5 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD5", DRIVE_LTO5, "[ULT3580-TD5]" ), /* IBM Ultrium Gen 5 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH5", DRIVE_LTO5_HH, "[ULTRIUM-HH5]" ), /* IBM Ultrium Gen 5 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH5", DRIVE_LTO5_HH, "[ULT3580-HH5]" ), /* IBM Ultrium Gen 5 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 5", DRIVE_LTO5_HH, "[HH LTO Gen 5]" ), /* IBM Ultrium Gen 5 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD6", DRIVE_LTO6, "[ULTRIUM-TD6]" ), /* IBM Ultrium Gen 6 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD6", DRIVE_LTO6, "[ULT3580-TD6]" ), /* IBM Ultrium Gen 6 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH6", DRIVE_LTO6_HH, "[ULTRIUM-HH6]" ), /* IBM Ultrium Gen 6 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH6", DRIVE_LTO6_HH, "[ULT3580-HH6]" ), /* IBM Ultrium Gen 6 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 6", DRIVE_LTO6_HH, "[HH LTO Gen 6]" ), /* IBM Ultrium Gen 6 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD7", DRIVE_LTO7, "[ULTRIUM-TD7]" ), /* IBM Ultrium Gen 7 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD7", DRIVE_LTO7, "[ULT3580-TD7]" ), /* IBM Ultrium Gen 7 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH7", DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* IBM Ultrium Gen 7 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH7", DRIVE_LTO7_HH, "[ULT3580-HH7]" ), /* IBM Ultrium Gen 7 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 7", DRIVE_LTO7_HH, "[HH LTO Gen 7]" ), /* IBM Ultrium Gen 7 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD8", DRIVE_LTO8, "[ULTRIUM-TD8]" ), /* IBM Ultrium Gen 8 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD8", DRIVE_LTO8, "[ULT3580-TD8]" ), /* IBM Ultrium Gen 8 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH8", DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* IBM Ultrium Gen 8 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH8", DRIVE_LTO8_HH, "[ULT3580-HH8]" ), /* IBM Ultrium Gen 8 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 8", DRIVE_LTO8_HH, "[HH LTO Gen 8]" ), /* IBM Ultrium Gen 8 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD9", DRIVE_LTO9, "[ULTRIUM-TD9]" ), /* IBM Ultrium Gen 9 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD9", DRIVE_LTO9, "[ULT3580-TD9]" ), /* IBM Ultrium Gen 9 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH9", DRIVE_LTO9_HH, "[ULTRIUM-HH9]" ), /* IBM Ultrium Gen 9 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH9", DRIVE_LTO9_HH, "[ULT3580-HH9]" ), /* IBM Ultrium Gen 9 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 9", DRIVE_LTO9_HH, "[HH LTO Gen 9]" ), /* IBM Ultrium Gen 9 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "03592E07", DRIVE_TS1140, "[03592E07]" ), /* IBM TS1140 */ - TAPEDRIVE( IBM_VENDOR_ID, "03592E08", DRIVE_TS1150, "[03592E08]" ), /* IBM TS1150 */ - TAPEDRIVE( IBM_VENDOR_ID, "0359255F", DRIVE_TS1155, "[0359255F]" ), /* IBM TS1155 */ - TAPEDRIVE( IBM_VENDOR_ID, "0359255E", DRIVE_TS1155, "[0359255E]" ), /* IBM TS1155 */ - TAPEDRIVE( IBM_VENDOR_ID, "0359260F", DRIVE_TS1160, "[0359260F]" ), /* IBM TS1160 */ - TAPEDRIVE( IBM_VENDOR_ID, "0359260E", DRIVE_TS1160, "[0359260E]" ), /* IBM TS1160 */ - TAPEDRIVE( IBM_VENDOR_ID, "0359260S", DRIVE_TS1160, "[0359260S]" ), /* IBM TS1160 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD5", VENDOR_IBM, DRIVE_LTO5, "[ULTRIUM-TD5]" ), /* IBM Ultrium Gen 5 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD5", VENDOR_IBM, DRIVE_LTO5, "[ULT3580-TD5]" ), /* IBM Ultrium Gen 5 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH5", VENDOR_IBM, DRIVE_LTO5_HH, "[ULTRIUM-HH5]" ), /* IBM Ultrium Gen 5 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH5", VENDOR_IBM, DRIVE_LTO5_HH, "[ULT3580-HH5]" ), /* IBM Ultrium Gen 5 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 5", VENDOR_IBM, DRIVE_LTO5_HH, "[HH LTO Gen 5]" ), /* IBM Ultrium Gen 5 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD6", VENDOR_IBM, DRIVE_LTO6, "[ULTRIUM-TD6]" ), /* IBM Ultrium Gen 6 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD6", VENDOR_IBM, DRIVE_LTO6, "[ULT3580-TD6]" ), /* IBM Ultrium Gen 6 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH6", VENDOR_IBM, DRIVE_LTO6_HH, "[ULTRIUM-HH6]" ), /* IBM Ultrium Gen 6 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH6", VENDOR_IBM, DRIVE_LTO6_HH, "[ULT3580-HH6]" ), /* IBM Ultrium Gen 6 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 6", VENDOR_IBM, DRIVE_LTO6_HH, "[HH LTO Gen 6]" ), /* IBM Ultrium Gen 6 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD7", VENDOR_IBM, DRIVE_LTO7, "[ULTRIUM-TD7]" ), /* IBM Ultrium Gen 7 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD7", VENDOR_IBM, DRIVE_LTO7, "[ULT3580-TD7]" ), /* IBM Ultrium Gen 7 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH7", VENDOR_IBM, DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* IBM Ultrium Gen 7 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH7", VENDOR_IBM, DRIVE_LTO7_HH, "[ULT3580-HH7]" ), /* IBM Ultrium Gen 7 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 7", VENDOR_IBM, DRIVE_LTO7_HH, "[HH LTO Gen 7]" ), /* IBM Ultrium Gen 7 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD8", VENDOR_IBM, DRIVE_LTO8, "[ULTRIUM-TD8]" ), /* IBM Ultrium Gen 8 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD8", VENDOR_IBM, DRIVE_LTO8, "[ULT3580-TD8]" ), /* IBM Ultrium Gen 8 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH8", VENDOR_IBM, DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* IBM Ultrium Gen 8 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH8", VENDOR_IBM, DRIVE_LTO8_HH, "[ULT3580-HH8]" ), /* IBM Ultrium Gen 8 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 8", VENDOR_IBM, DRIVE_LTO8_HH, "[HH LTO Gen 8]" ), /* IBM Ultrium Gen 8 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-TD9", VENDOR_IBM, DRIVE_LTO9, "[ULTRIUM-TD9]" ), /* IBM Ultrium Gen 9 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD9", VENDOR_IBM, DRIVE_LTO9, "[ULT3580-TD9]" ), /* IBM Ultrium Gen 9 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH9", VENDOR_IBM, DRIVE_LTO9_HH, "[ULTRIUM-HH9]" ), /* IBM Ultrium Gen 9 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH9", VENDOR_IBM, DRIVE_LTO9_HH, "[ULT3580-HH9]" ), /* IBM Ultrium Gen 9 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "HH LTO Gen 9", VENDOR_IBM, DRIVE_LTO9_HH, "[HH LTO Gen 9]" ), /* IBM Ultrium Gen 9 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "03592E07", VENDOR_IBM, DRIVE_TS1140, "[03592E07]" ), /* IBM TS1140 */ + TAPEDRIVE( IBM_VENDOR_ID, "03592E08", VENDOR_IBM, DRIVE_TS1150, "[03592E08]" ), /* IBM TS1150 */ + TAPEDRIVE( IBM_VENDOR_ID, "0359255F", VENDOR_IBM, DRIVE_TS1155, "[0359255F]" ), /* IBM TS1155 */ + TAPEDRIVE( IBM_VENDOR_ID, "0359255E", VENDOR_IBM, DRIVE_TS1155, "[0359255E]" ), /* IBM TS1155 */ + TAPEDRIVE( IBM_VENDOR_ID, "0359260F", VENDOR_IBM, DRIVE_TS1160, "[0359260F]" ), /* IBM TS1160 */ + TAPEDRIVE( IBM_VENDOR_ID, "0359260E", VENDOR_IBM, DRIVE_TS1160, "[0359260E]" ), /* IBM TS1160 */ + TAPEDRIVE( IBM_VENDOR_ID, "0359260S", VENDOR_IBM, DRIVE_TS1160, "[0359260S]" ), /* IBM TS1160 */ /* End of supported_devices */ NULL }; struct supported_device *usb_supported_drives[] = { - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD5", DRIVE_LTO5, "[ULT3580-TD5]" ), /* IBM Ultrium Gen 5 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH5", DRIVE_LTO5_HH, "[ULTRIUM-HH5]" ), /* IBM Ultrium Gen 5 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH5", DRIVE_LTO5_HH, "[ULT3580-HH5]" ), /* IBM Ultrium Gen 5 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD6", DRIVE_LTO6, "[ULT3580-TD6]" ), /* IBM Ultrium Gen 6 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH6", DRIVE_LTO6_HH, "[ULTRIUM-HH6]" ), /* IBM Ultrium Gen 6 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH6", DRIVE_LTO6_HH, "[ULT3580-HH6]" ), /* IBM Ultrium Gen 6 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD7", DRIVE_LTO7, "[ULT3580-TD7]" ), /* IBM Ultrium Gen 7 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH7", DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* IBM Ultrium Gen 7 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH7", DRIVE_LTO7_HH, "[ULT3580-HH7]" ), /* IBM Ultrium Gen 7 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD8", DRIVE_LTO8, "[ULT3580-TD8]" ), /* IBM Ultrium Gen 8 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH8", DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* IBM Ultrium Gen 8 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH8", DRIVE_LTO8_HH, "[ULT3580-HH8]" ), /* IBM Ultrium Gen 8 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD9", DRIVE_LTO9, "[ULT3580-TD9]" ), /* IBM Ultrium Gen 9 */ - TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH9", DRIVE_LTO9_HH, "[ULTRIUM-HH9]" ), /* IBM Ultrium Gen 9 Half-High */ - TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH9", DRIVE_LTO9_HH, "[ULT3580-HH9]" ), /* IBM Ultrium Gen 9 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD5", VENDOR_IBM, DRIVE_LTO5, "[ULT3580-TD5]" ), /* IBM Ultrium Gen 5 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH5", VENDOR_IBM, DRIVE_LTO5_HH, "[ULTRIUM-HH5]" ), /* IBM Ultrium Gen 5 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH5", VENDOR_IBM, DRIVE_LTO5_HH, "[ULT3580-HH5]" ), /* IBM Ultrium Gen 5 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD6", VENDOR_IBM, DRIVE_LTO6, "[ULT3580-TD6]" ), /* IBM Ultrium Gen 6 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH6", VENDOR_IBM, DRIVE_LTO6_HH, "[ULTRIUM-HH6]" ), /* IBM Ultrium Gen 6 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH6", VENDOR_IBM, DRIVE_LTO6_HH, "[ULT3580-HH6]" ), /* IBM Ultrium Gen 6 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD7", VENDOR_IBM, DRIVE_LTO7, "[ULT3580-TD7]" ), /* IBM Ultrium Gen 7 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH7", VENDOR_IBM, DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* IBM Ultrium Gen 7 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH7", VENDOR_IBM, DRIVE_LTO7_HH, "[ULT3580-HH7]" ), /* IBM Ultrium Gen 7 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD8", VENDOR_IBM, DRIVE_LTO8, "[ULT3580-TD8]" ), /* IBM Ultrium Gen 8 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH8", VENDOR_IBM, DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* IBM Ultrium Gen 8 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH8", VENDOR_IBM, DRIVE_LTO8_HH, "[ULT3580-HH8]" ), /* IBM Ultrium Gen 8 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-TD9", VENDOR_IBM, DRIVE_LTO9, "[ULT3580-TD9]" ), /* IBM Ultrium Gen 9 */ + TAPEDRIVE( IBM_VENDOR_ID, "ULTRIUM-HH9", VENDOR_IBM, DRIVE_LTO9_HH, "[ULTRIUM-HH9]" ), /* IBM Ultrium Gen 9 Half-High */ + TAPEDRIVE( IBM_VENDOR_ID, "ULT3580-HH9", VENDOR_IBM, DRIVE_LTO9_HH, "[ULT3580-HH9]" ), /* IBM Ultrium Gen 9 Half-High */ /* End of supported_devices */ NULL }; diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index f5830792..71b53c5c 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -487,12 +487,12 @@ static int _raw_open(struct sg_data *priv) priv->dev.fd = -1; return ret; } - priv->vendor = get_vendor_id(id_data.vendor_id); struct supported_device **cur = get_supported_devs(priv->vendor); while(*cur) { if((! strncmp(id_data.vendor_id, (*cur)->vendor_id, strlen((*cur)->vendor_id)) ) && (! strncmp(id_data.product_id, (*cur)->product_id, strlen((*cur)->product_id)) ) ) { + priv->vendor = (*cur)->vendor_type; drive_type = (*cur)->drive_type; break; } @@ -2969,7 +2969,8 @@ int sg_remaining_capacity(void *device, struct tc_remaining_cap *cap) memset(buffer, 0, LOGSENSEPAGE); if ((IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) || - (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))) { + (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06)) || + (priv->vendor == VENDOR_QUANTUM_B && IS_LTO(priv->drive_type))) { /* Use LogPage 0x31 */ ret = sg_logsense(device, (uint8_t)LOG_TAPECAPACITY, (uint8_t)0, (void *)buffer, LOGSENSEPAGE); diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index 379842b9..52b9abd0 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -856,14 +856,12 @@ int iokit_open(const char *devname, void **handle) } strncpy(priv->drive_serial, id_data.unit_serial, UNIT_SERIAL_LENGTH - 1); - /* Convert vendor_id in inq buffer to integer based id */ - priv->vendor = get_vendor_id(id_data.vendor_id); - /* Check the drive is supportable */ struct supported_device **cur = get_supported_devs(priv->vendor); while(cur && *cur) { if((! strncmp(id_data.vendor_id, (*cur)->vendor_id, strlen((*cur)->vendor_id)) ) && (! strncmp(id_data.product_id, (*cur)->product_id, strlen((*cur)->product_id)) ) ) { + priv->vendor = (*cur)->vendor_type; drive_type = (*cur)->drive_type; break; } @@ -2315,7 +2313,8 @@ int iokit_remaining_capacity(void *device, struct tc_remaining_cap *cap) memset(&buffer, 0, LOGSENSEPAGE); if ((IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) || - (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06))) { + (priv->vendor == VENDOR_HP && IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x06)) || + (priv->vendor == VENDOR_QUANTUM_B && IS_LTO(priv->drive_type))) { /* Use LogPage 0x31 */ ret = iokit_logsense(device, (uint8_t)LOG_TAPECAPACITY, (uint8_t)0, (void *)buffer, LOGSENSEPAGE); diff --git a/src/tape_drivers/quantum_tape.c b/src/tape_drivers/quantum_tape.c index 880afbff..8ca47914 100644 --- a/src/tape_drivers/quantum_tape.c +++ b/src/tape_drivers/quantum_tape.c @@ -63,12 +63,12 @@ #include "libltfs/ltfs_endian.h" struct supported_device *quantum_supported_drives[] = { - TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH5", DRIVE_LTO5_HH, "[ULTRIUM-HH5]" ), /* QUANTUM Ultrium Gen 5 Half-High */ - TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH6", DRIVE_LTO6_HH, "[ULTRIUM-HH6]" ), /* QUANTUM Ultrium Gen 6 Half-High */ - TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH7", DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* QUANTUM Ultrium Gen 7 Half-High */ - TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH8", DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* QUANTUM Ultrium Gen 8 Half-High */ - TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 5", DRIVE_LTO5_HH, "[ULTRIUM-5]" ), /* Another QUANTUM Ultrium Gen 5 Half-High */ - TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 6", DRIVE_LTO6_HH, "[ULTRIUM-6]" ), /* Another QUANTUM Ultrium Gen 6 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH5", VENDOR_QUANTUM, DRIVE_LTO5_HH, "[ULTRIUM-HH5]" ), /* QUANTUM Ultrium Gen 5 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH6", VENDOR_QUANTUM, DRIVE_LTO6_HH, "[ULTRIUM-HH6]" ), /* QUANTUM Ultrium Gen 6 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH7", VENDOR_QUANTUM, DRIVE_LTO7_HH, "[ULTRIUM-HH7]" ), /* QUANTUM Ultrium Gen 7 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM-HH8", VENDOR_QUANTUM, DRIVE_LTO8_HH, "[ULTRIUM-HH8]" ), /* QUANTUM Ultrium Gen 8 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 5", VENDOR_QUANTUM_B, DRIVE_LTO5_HH, "[ULTRIUM-5]" ), /* Another QUANTUM Ultrium Gen 5 Half-High */ + TAPEDRIVE( QUANTUM_VENDOR_ID, "ULTRIUM 6", VENDOR_QUANTUM_B, DRIVE_LTO6_HH, "[ULTRIUM-6]" ), /* Another QUANTUM Ultrium Gen 6 Half-High */ /* End of supported_devices */ NULL }; diff --git a/src/tape_drivers/tape_drivers.h b/src/tape_drivers/tape_drivers.h index a9d7bf5e..6ca2ce93 100644 --- a/src/tape_drivers/tape_drivers.h +++ b/src/tape_drivers/tape_drivers.h @@ -149,11 +149,12 @@ static inline int _sense2errorcode(uint32_t sense, struct error_table *table, ch struct supported_device { char vendor_id[VENDOR_ID_LENGTH + 1]; char product_id[PRODUCT_ID_LENGTH + 1]; + int vendor_type; int drive_type; char product_name[PRODUCT_NAME_LENGTH + 1]; }; -#define TAPEDRIVE(v, p, t, n) &(struct supported_device){ v, p, t, n } +#define TAPEDRIVE(v, p, vt, dt, n) &(struct supported_device){ v, p, vt, dt, n } #define TAPE_FAMILY_MASK (0xF000) #define TAPE_FAMILY_ENTERPRISE (0x1000) @@ -177,7 +178,8 @@ enum { VENDOR_UNKNOWN = 0, VENDOR_IBM, VENDOR_HP, - VENDOR_QUANTUM, + VENDOR_QUANTUM, /* Quantum model C */ + VENDOR_QUANTUM_B, /* Quantum model B */ }; enum { diff --git a/src/tape_drivers/vendor_compat.c b/src/tape_drivers/vendor_compat.c index 7a0acbb1..fa5e08a2 100644 --- a/src/tape_drivers/vendor_compat.c +++ b/src/tape_drivers/vendor_compat.c @@ -281,20 +281,6 @@ struct error_table standard_tape_errors[] = { {0xFFFFFF, -EDEV_UNKNOWN, "Unknown Error code"}, }; -int get_vendor_id(char* vendor) -{ - if (!strncmp(vendor, IBM_VENDOR_ID, strlen(IBM_VENDOR_ID))) - return VENDOR_IBM; - else if (!strncmp(vendor, HP_VENDOR_ID, strlen(HP_VENDOR_ID))) - return VENDOR_HP; - else if (!strncmp(vendor, HPE_VENDOR_ID, strlen(HPE_VENDOR_ID))) - return VENDOR_HP; - else if (!strncmp(vendor, QUANTUM_VENDOR_ID, strlen(QUANTUM_VENDOR_ID))) - return VENDOR_QUANTUM; - else - return VENDOR_UNKNOWN; -} - struct supported_device **get_supported_devs(int vendor) { struct supported_device **cur = NULL; diff --git a/src/tape_drivers/vendor_compat.h b/src/tape_drivers/vendor_compat.h index e6eec141..93a509ff 100644 --- a/src/tape_drivers/vendor_compat.h +++ b/src/tape_drivers/vendor_compat.h @@ -65,7 +65,6 @@ extern "C" { extern struct error_table standard_tape_errors[]; -int get_vendor_id(char* vendor); struct supported_device **get_supported_devs(int vendor); bool drive_has_supported_fw(int vendor, int drive_type, const unsigned char * const revision); unsigned char assume_cart_type(const unsigned char dc); From f0575354f60839c1cc0f2774aae8dc41af85b514 Mon Sep 17 00:00:00 2001 From: piste2750 <32239919+piste2750@users.noreply.github.com> Date: Tue, 22 Jun 2021 10:33:59 +0900 Subject: [PATCH 039/121] Fix bugs to fetch supported drive list (#298) --- src/tape_drivers/freebsd/cam/cam_tc.c | 2 +- src/tape_drivers/linux/sg/sg_tape.c | 2 +- src/tape_drivers/osx/iokit/iokit_tape.c | 2 +- src/tape_drivers/vendor_compat.c | 21 +++++++++------------ src/tape_drivers/vendor_compat.h | 2 +- 5 files changed, 13 insertions(+), 16 deletions(-) diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 3e2a4ded..6fdb2582 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -279,7 +279,7 @@ int camtape_open(const char *devname, void **handle) ltfsmsg(LTFS_INFO, 31229I, vendor); /* Check the drive is supportable */ - struct supported_device **cur = get_supported_devs(softc->vendor); + struct supported_device **cur = get_supported_devs((char *)softc->cd->inq_data.vendor); while(*cur) { if ((! strncmp((char*)softc->cd->inq_data.vendor, (*cur)->vendor_id, strlen((*cur)->vendor_id)) ) && (! strncmp((char*)softc->cd->inq_data.product, (*cur)->product_id, strlen((*cur)->product_id)) ) ) { diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index 71b53c5c..be06b55f 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -488,7 +488,7 @@ static int _raw_open(struct sg_data *priv) return ret; } - struct supported_device **cur = get_supported_devs(priv->vendor); + struct supported_device **cur = get_supported_devs(id_data.vendor_id); while(*cur) { if((! strncmp(id_data.vendor_id, (*cur)->vendor_id, strlen((*cur)->vendor_id)) ) && (! strncmp(id_data.product_id, (*cur)->product_id, strlen((*cur)->product_id)) ) ) { diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index 52b9abd0..91d2c0b6 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -857,7 +857,7 @@ int iokit_open(const char *devname, void **handle) strncpy(priv->drive_serial, id_data.unit_serial, UNIT_SERIAL_LENGTH - 1); /* Check the drive is supportable */ - struct supported_device **cur = get_supported_devs(priv->vendor); + struct supported_device **cur = get_supported_devs(id_data.vendor_id); while(cur && *cur) { if((! strncmp(id_data.vendor_id, (*cur)->vendor_id, strlen((*cur)->vendor_id)) ) && (! strncmp(id_data.product_id, (*cur)->product_id, strlen((*cur)->product_id)) ) ) { diff --git a/src/tape_drivers/vendor_compat.c b/src/tape_drivers/vendor_compat.c index fa5e08a2..c35bd8d1 100644 --- a/src/tape_drivers/vendor_compat.c +++ b/src/tape_drivers/vendor_compat.c @@ -281,21 +281,18 @@ struct error_table standard_tape_errors[] = { {0xFFFFFF, -EDEV_UNKNOWN, "Unknown Error code"}, }; -struct supported_device **get_supported_devs(int vendor) +struct supported_device **get_supported_devs(const char *vendor_str) { struct supported_device **cur = NULL; - switch (vendor) { - case VENDOR_IBM: - cur = ibm_supported_drives; - break; - case VENDOR_HP: - cur = hp_supported_drives; - break; - case VENDOR_QUANTUM: - cur = quantum_supported_drives; - break; - } + if (! strncmp(vendor_str, IBM_VENDOR_ID, strlen(IBM_VENDOR_ID))) + cur = ibm_supported_drives; + else if (! strncmp(vendor_str, HP_VENDOR_ID, strlen(IBM_VENDOR_ID))) + cur = hp_supported_drives; + else if (! strncmp(vendor_str, HPE_VENDOR_ID, strlen(IBM_VENDOR_ID))) + cur = hp_supported_drives; + else if (! strncmp(vendor_str, QUANTUM_VENDOR_ID, strlen(IBM_VENDOR_ID))) + cur = quantum_supported_drives; return cur; } diff --git a/src/tape_drivers/vendor_compat.h b/src/tape_drivers/vendor_compat.h index 93a509ff..19ae377f 100644 --- a/src/tape_drivers/vendor_compat.h +++ b/src/tape_drivers/vendor_compat.h @@ -65,7 +65,7 @@ extern "C" { extern struct error_table standard_tape_errors[]; -struct supported_device **get_supported_devs(int vendor); +struct supported_device **get_supported_devs(const char *vendor_str); bool drive_has_supported_fw(int vendor, int drive_type, const unsigned char * const revision); unsigned char assume_cart_type(const unsigned char dc); int is_supported_tape(unsigned char type, unsigned char density, bool *is_worm); From ab7d0f09ba5f133966c077e81eab23eb5876238a Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 22 Jun 2021 16:08:40 +0900 Subject: [PATCH 040/121] Experimental support of Tandberg drives (#299) --- src/tape_drivers/hp_tape.c | 2 ++ src/tape_drivers/hp_tape.h | 5 +++-- src/tape_drivers/vendor_compat.c | 2 ++ 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/src/tape_drivers/hp_tape.c b/src/tape_drivers/hp_tape.c index b812ab69..365ce4a4 100644 --- a/src/tape_drivers/hp_tape.c +++ b/src/tape_drivers/hp_tape.c @@ -67,6 +67,8 @@ struct supported_device *hp_supported_drives[] = { TAPEDRIVE( HP_VENDOR_ID, "Ultrium 6-SCSI", VENDOR_HP, DRIVE_LTO6, "[Ultrium 6-SCSI]" ), /* HP Ultrium Gen 6 */ TAPEDRIVE( HP_VENDOR_ID, "Ultrium 7-SCSI", VENDOR_HP, DRIVE_LTO7, "[Ultrium 7-SCSI]" ), /* HP Ultrium Gen 7 */ TAPEDRIVE( HPE_VENDOR_ID, "Ultrium 8-SCSI", VENDOR_HP, DRIVE_LTO8, "[Ultrium 8-SCSI]" ), /* HPE Ultrium Gen 8 */ + TAPEDRIVE( TANDBERG_VENDOR_ID, "LTO-5 HH", VENDOR_HP, DRIVE_LTO5_HH, "[TANDBERG LTO5]" ), /* TANDBERG LTO-5 HH */ + TAPEDRIVE( TANDBERG_VENDOR_ID, "LTO-6 HH", VENDOR_HP, DRIVE_LTO6_HH, "[TANDBERG LTO6]" ), /* TANDBERG LTO-6 HH */ /* End of supported_devices */ NULL }; diff --git a/src/tape_drivers/hp_tape.h b/src/tape_drivers/hp_tape.h index 42515110..5af848ec 100644 --- a/src/tape_drivers/hp_tape.h +++ b/src/tape_drivers/hp_tape.h @@ -65,8 +65,9 @@ extern "C" { #endif -#define HP_VENDOR_ID "HP" -#define HPE_VENDOR_ID "HPE" +#define HP_VENDOR_ID "HP" +#define HPE_VENDOR_ID "HPE" +#define TANDBERG_VENDOR_ID "TANDBERG" extern struct error_table hp_tape_errors[]; diff --git a/src/tape_drivers/vendor_compat.c b/src/tape_drivers/vendor_compat.c index c35bd8d1..59dffa04 100644 --- a/src/tape_drivers/vendor_compat.c +++ b/src/tape_drivers/vendor_compat.c @@ -291,6 +291,8 @@ struct supported_device **get_supported_devs(const char *vendor_str) cur = hp_supported_drives; else if (! strncmp(vendor_str, HPE_VENDOR_ID, strlen(IBM_VENDOR_ID))) cur = hp_supported_drives; + else if (! strncmp(vendor_str, TANDBERG_VENDOR_ID, strlen(IBM_VENDOR_ID))) + cur = hp_supported_drives; else if (! strncmp(vendor_str, QUANTUM_VENDOR_ID, strlen(IBM_VENDOR_ID))) cur = quantum_supported_drives; From 0fae6d816c353cf82e2eecf7970649627ebcb21c Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 25 Jun 2021 15:43:24 +0900 Subject: [PATCH 041/121] Correct compare length of vendor IDs (#301) --- src/tape_drivers/vendor_compat.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/tape_drivers/vendor_compat.c b/src/tape_drivers/vendor_compat.c index 59dffa04..7096415b 100644 --- a/src/tape_drivers/vendor_compat.c +++ b/src/tape_drivers/vendor_compat.c @@ -287,13 +287,13 @@ struct supported_device **get_supported_devs(const char *vendor_str) if (! strncmp(vendor_str, IBM_VENDOR_ID, strlen(IBM_VENDOR_ID))) cur = ibm_supported_drives; - else if (! strncmp(vendor_str, HP_VENDOR_ID, strlen(IBM_VENDOR_ID))) + else if (! strncmp(vendor_str, HP_VENDOR_ID, strlen(HP_VENDOR_ID))) cur = hp_supported_drives; - else if (! strncmp(vendor_str, HPE_VENDOR_ID, strlen(IBM_VENDOR_ID))) + else if (! strncmp(vendor_str, HPE_VENDOR_ID, strlen(HPE_VENDOR_ID))) cur = hp_supported_drives; - else if (! strncmp(vendor_str, TANDBERG_VENDOR_ID, strlen(IBM_VENDOR_ID))) + else if (! strncmp(vendor_str, TANDBERG_VENDOR_ID, strlen(TANDBERG_VENDOR_ID))) cur = hp_supported_drives; - else if (! strncmp(vendor_str, QUANTUM_VENDOR_ID, strlen(IBM_VENDOR_ID))) + else if (! strncmp(vendor_str, QUANTUM_VENDOR_ID, strlen(QUANTUM_VENDOR_ID))) cur = quantum_supported_drives; return cur; From 18ad9cd3eebed7b5f2b7c296b681991b6cd17ab5 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 4 Aug 2021 13:35:00 +0900 Subject: [PATCH 042/121] Log the generation of index at mount/unmount (#304) --- messages/libltfs/root.txt | 4 +- src/libltfs/ltfs.c | 87 +++++++++++++++++++++------------------ 2 files changed, 48 insertions(+), 43 deletions(-) diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index 4e0afe47..4b8c5c20 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -124,7 +124,7 @@ root:table { 11029E:string { "Cannot mount volume: failed to save the append position for the index partition." } 11030I:string { "Failed to sync volume (%d). Stop the periodic sync thread." } //unused 11030E:string { "Cannot release the device lock (%s)." } - //11031D:string { "Volume mounted successfully." } + 11031I:string { "Volume mounted successfully %s (Gen = %lld, (%c, %lld) -> (%c, %lld), %s)." } 11032D:string { "Unmounting the volume." } 11033E:string { "Cannot unmount: failed to write an index." } 11034I:string { "Volume unmounted successfully." } @@ -778,7 +778,7 @@ v 17233E:string { "Failed to kick gcore." } 17234W:string { "The index read from the tape uses a newer version of the LTFS format than the one supported by this software. If this tape is modified, the index downgrades to format version %s from %d.%d.%d." } 17235I:string { "Writing index of %s to %c (Reason: %s, %lld files) %s." } - 17236I:string { "Wrote index of %s (%c, %s)." } + 17236I:string { "Wrote index of %s (Gen = %lld, Part = %c, Pos = %lld, %s)." } 17237E:string { "WORM related error (%s)." } 17238I:string { "WORM status updated (%s=>%d) '%s'." } 17239E:string { "Failed to update density (%s) %d." } diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 5c865845..6263c68d 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -115,6 +115,17 @@ const char *ltfs_format_version() return LTFS_INDEX_VERSION_STR; } +static inline char* _get_barcode(struct ltfs_volume *vol) +{ + char *barcode = NULL; + + if (vol->label->barcode[0] != ' ') + barcode = vol->label->barcode; + else + barcode = LTFS_NO_BARCODE; + + return barcode; +} /** * Initialize the LTFS functions, currently the XML parser and the logging component. @@ -1473,6 +1484,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec /* TODO: is_worm_recovery_mount should be set by user via option */ int vollock = UNLOCKED_MAM; char *vl_print = NULL; + char *barcode = NULL; ltfsmsg(LTFS_INFO, 11005I); @@ -1782,6 +1794,17 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec vol->lock_status = vol->index->vollock; } + barcode = _get_barcode(vol); + + ltfsmsg(LTFS_INFO, 11031I, + barcode, + (unsigned long long)vol->index->generation, + vol->index->selfptr.partition, + (unsigned long long)vol->index->selfptr.block, + vol->index->backptr.partition, + (unsigned long long)vol->index->backptr.block, + tape_get_serialnumber(vol->device)); + out_unlock: if (index && vol->index) ltfs_index_free(&index); @@ -1841,10 +1864,7 @@ void ltfs_set_index_dirty(bool locking, bool atime, struct ltfs_index *idx) ltfs_mutex_unlock(&idx->dirty_lock); if (!was_dirty && idx->dirty) { - if (idx->root->vol->label->barcode[0] != ' ') - ltfsmsg(LTFS_INFO, 11337I, true, idx->root->vol->label->barcode, idx->root->vol); - else - ltfsmsg(LTFS_INFO, 11337I, true, LTFS_NO_BARCODE, idx->root->vol); + ltfsmsg(LTFS_INFO, 11337I, true, _get_barcode(idx->root->vol), idx->root->vol); } } } @@ -1870,10 +1890,7 @@ void ltfs_unset_index_dirty(bool update_version, struct ltfs_index *idx) ltfs_mutex_unlock(&idx->dirty_lock); if (was_dirty && !idx->dirty) { - if (idx->root->vol->label->barcode[0] != ' ') - ltfsmsg(LTFS_INFO, 11337I, false, idx->root->vol->label->barcode, idx->root->vol); - else - ltfsmsg(LTFS_INFO, 11337I, false, LTFS_NO_BARCODE, idx->root->vol); + ltfsmsg(LTFS_INFO, 11337I, false, _get_barcode(idx->root->vol), idx->root->vol); } } } @@ -2176,7 +2193,7 @@ void ltfs_set_work_dir(const char *dir, struct ltfs_volume *vol) /** * Configure EOD (end of data) checking for a volume. This should be done before - * calling ltfs_mount. + * calling ltfs_mount * The EOD check is enabled by default. */ void ltfs_set_eod_check(bool use, struct ltfs_volume *vol) @@ -2289,10 +2306,7 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) return ret; } - if (vol->label->barcode[0] == ' ' || vol->label->barcode[0] == '\0') - bc_print = LTFS_NO_BARCODE; - else - bc_print = vol->label->barcode; + bc_print = _get_barcode(vol); if (write_perm) { ltfsmsg(LTFS_INFO, 11343I, bc_print); @@ -2470,7 +2484,12 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) * ignore failures when updating MAM parameters. */ ltfs_update_cart_coherency(vol); - ltfsmsg(LTFS_INFO, 17236I, bc_print, partition, tape_get_serialnumber(vol->device)); + ltfsmsg(LTFS_INFO, 17236I, + bc_print, + (unsigned long long)vol->index->generation, + vol->index->selfptr.partition, + (unsigned long long)vol->index->selfptr.block, + tape_get_serialnumber(vol->device)); /* update append position */ if (partition == ltfs_ip_id(vol)) { @@ -2552,13 +2571,9 @@ int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, return -ENOMEM; } - if (vol->label->barcode[0] != ' ') { - ltfsmsg(LTFS_INFO, 17235I, vol->label->barcode, 'Z', "Volume Cache", - (unsigned long long)vol->index->file_count, path); - } else { - ltfsmsg(LTFS_INFO, 17235I, LTFS_NO_BARCODE, 'Z', "Volume Cache", - (unsigned long long)vol->index->file_count, path); - } + + ltfsmsg(LTFS_INFO, 17235I, _get_barcode(vol), 'Z', "Volume Cache", + (unsigned long long)vol->index->file_count, path); ret = xml_schema_to_file(path, vol->index->creator, reason, vol->index); if (ret < 0) { @@ -2574,10 +2589,12 @@ int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, ltfsmsg(LTFS_ERR, 17184E, errno); } - if (vol->label->barcode[0] != ' ') - ltfsmsg(LTFS_INFO, 17236I, vol->label->barcode, 'Z', path); - else - ltfsmsg(LTFS_INFO, 17236I, LTFS_NO_BARCODE, 'Z', path); + ltfsmsg(LTFS_INFO, 17236I, + _get_barcode(vol), + (unsigned long long)vol->index->generation, + 'Z', + (unsigned long long)vol->index->selfptr.block, + path); free(path); return ret; @@ -3160,10 +3177,7 @@ int ltfs_revalidate(bool have_write_lock, struct ltfs_volume *vol) CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); - if (vol->label->barcode[0] != ' ') - ltfsmsg(LTFS_INFO, 11312I, vol->label->barcode); - else - ltfsmsg(LTFS_INFO, 11312I, LTFS_NO_BARCODE); + ltfsmsg(LTFS_INFO, 11312I, _get_barcode(vol)); /* Block other libltfs operations until revalidation finishes */ ltfs_thread_mutex_lock(&vol->reval_lock); @@ -3352,15 +3366,9 @@ int ltfs_revalidate(bool have_write_lock, struct ltfs_volume *vol) releasewrite_mrsw(&vol->lock); if (ret < 0) { - if (vol->label->barcode[0] != ' ') - ltfsmsg(LTFS_ERR, 11313E, ret, vol->label->barcode); - else - ltfsmsg(LTFS_ERR, 11313E, ret, LTFS_NO_BARCODE); + ltfsmsg(LTFS_ERR, 11313E, ret, _get_barcode(vol)); } else { - if (vol->label->barcode[0] != ' ') - ltfsmsg(LTFS_INFO, 11340I, vol->label->barcode); - else - ltfsmsg(LTFS_INFO, 11340I, LTFS_NO_BARCODE); + ltfsmsg(LTFS_INFO, 11340I, _get_barcode(vol)); } return ret; @@ -3402,10 +3410,7 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) releaseread_mrsw(&vol->lock); if (dirty) { - if (vol->label->barcode[0] == ' ' || vol->label->barcode[0] == '\0') - bc_print = LTFS_NO_BARCODE; - else - bc_print = vol->label->barcode; + bc_print = _get_barcode(vol); ltfsmsg(LTFS_INFO, 11338I, bc_print, vol->device->serial_number); From e1bca53f346ad344a7b6f6c0982aeaadf3bd2b10 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Thu, 5 Aug 2021 18:40:53 +0900 Subject: [PATCH 043/121] Update the status of the spec 2.5 --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index d4ffeb87..7bce62d1 100644 --- a/README.md +++ b/README.md @@ -42,12 +42,12 @@ LTFS Format Specification is specified data placement, shape of index and names The table below show status of the LTFS format Specification - | Version | Status of SNIA | Status of ISO | + | Version | Status of SNIA | Status of ISO | |:-------:|:------------------------------------------------------------------------------------------------------------:|:-------------------------:| | 2.2 | [Published](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | Published as `20919:2016` | | 2.3.1 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.PDF) | - | | 2.4 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | - | - | 2.5.1 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2%205%201_Standard.pdf) | Under development | + | 2.5.1 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2%205%201_Standard.pdf) | Published as `20919:2021` | # How to use the LTFS (Quick start) From 4b2e7445340a99f89160c81f0e88c980a8a8d142 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 10 Aug 2021 10:55:59 +0900 Subject: [PATCH 044/121] Handle unupdated MAM tape at write perm (#307) --- messages/libltfs/root.txt | 11 ++++- src/libltfs/ltfs.c | 99 ++++++++++++++++++++++++++++++++++++--- 2 files changed, 102 insertions(+), 8 deletions(-) diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index 4b8c5c20..c36e1f77 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -124,7 +124,7 @@ root:table { 11029E:string { "Cannot mount volume: failed to save the append position for the index partition." } 11030I:string { "Failed to sync volume (%d). Stop the periodic sync thread." } //unused 11030E:string { "Cannot release the device lock (%s)." } - 11031I:string { "Volume mounted successfully %s (Gen = %lld, (%c, %lld) -> (%c, %lld), %s)." } + 11031I:string { "Volume mounted successfully. %s : Gen = %lld / (%c, %lld) -> (%c, %lld) / %s." } 11032D:string { "Unmounting the volume." } 11033E:string { "Cannot unmount: failed to write an index." } 11034I:string { "Volume unmounted successfully." } @@ -427,7 +427,7 @@ root:table { 11330I:string { "Loading cartridge." } 11331E:string { "Failed to load the cartridge (%s)." } 11332I:string { "Load successful." } - 11333I:string { "A cartridge with write-perm error is detected on %s. Seek the newest index (IP Gen = %llu, VCR = %llu) (DP Gen = %llu, VCR = %llu) (VCR = %llu)." } + 11333I:string { "A cartridge with write-perm error is detected on %s. Seek the newest index (IP: Gen = %llu, VCR = %llu) (DP: Gen = %llu, VCR = %llu) (VCR = %llu)." } 11334I:string { "Remove extent : %s (%llu, %llu)." } 11335D:string { "Get physical block position (%d - %d)." } 11336I:string { "The attribute does not exist. Ignore the expected error." } @@ -824,6 +824,13 @@ v 17280I:string { "Failed to get the length of the RAO input file: %s (%d)." } 17281I:string { "Failed to read from RAO input file: %s (%d)." } 17282I:string { "Read length from RAO input file is unexpected: %s (Actual %ld, Expected %ld)." } + 17283I:string { "Detected unmatched VCR value between MAM and VCR (%llu, %llu)." } + 17284I:string { "Seaching the final index in %s." } + 17285E:string { "Failed to search the final index in %s (%d)." } + 17286I:string { "VCR value is matched between %s-MAM and VCR (%llu)." } + 17287I:string { "Making R/O mount from the location (%c, %llu)." } + 17288I:string { "Detected the final indexes (IP: Gen = %llu, Pos = %llu) (DP: Gen = %llu, Pos = %llu)." } + 17289I:string { "Skip parsing the final index on IP." } // For Debug 19999I:string { "%s %s %d." } diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 6263c68d..2f05a5f5 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1461,6 +1461,51 @@ int ltfs_start_mount(bool trial, struct ltfs_volume *vol) return 0; } +static inline int _ltfs_search_index_wp(bool recover_symlink, bool can_skip_ip, + struct tc_position *seekpos, struct ltfs_volume *vol) +{ + int ret = 0; + tape_block_t end_pos, index_end_pos; + bool fm_after, blocks_after; + + ltfsmsg(LTFS_INFO, 17284I, "IP"); + ret = ltfs_seek_index(vol->label->partid_ip, &end_pos, &index_end_pos, &fm_after, + &blocks_after, recover_symlink, vol); + if (ret) { + if (can_skip_ip) { + ltfsmsg(LTFS_INFO, 17289I); + vol->ip_coh.count = 0; + vol->ip_coh.set_id = 0; + } else { + ltfsmsg(LTFS_ERR, 17285E, "IP", ret); + return -LTFS_INDEX_INVALID; + } + } + + ltfsmsg(LTFS_INFO, 17284I, "DP"); + ret = ltfs_seek_index(vol->label->partid_dp, &end_pos, &index_end_pos, &fm_after, + &blocks_after, recover_symlink, vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17285E, "DP", ret); + return -LTFS_INDEX_INVALID; + } + + /* Use the latest index on the tape */ + ltfsmsg(LTFS_INFO, 17288I, + (unsigned long long)vol->ip_coh.count, (unsigned long long)vol->ip_coh.set_id, + (unsigned long long)vol->dp_coh.count, (unsigned long long)vol->dp_coh.set_id); + + if (vol->ip_coh.count > vol->dp_coh.count) { + seekpos->partition = ltfs_part_id2num(vol->label->partid_ip, vol); + seekpos->block = vol->ip_coh.set_id; + } else { + seekpos->partition = ltfs_part_id2num(vol->label->partid_dp, vol); + seekpos->block = vol->dp_coh.set_id; + } + + return 0; +} + /** * Read LTFS data structures from a tape, checking for consistency (and restoring it * if possible). This function doesn't bother locking vol->index, as it must complete before any @@ -1602,8 +1647,26 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec */ ltfsmsg(LTFS_INFO, 17264I, "DP", vl_print); } - seekpos.partition = ltfs_part_id2num(vol->label->partid_dp, vol); - seekpos.block = vol->dp_coh.set_id; + + if (volume_change_ref != vol->dp_coh.volume_change_ref) { + /* + * Cannot trust the index info on MAM, search the last indexes + * This would happen when the drive returns an error against acquiring the VCR + * while write error handling. + */ + ltfsmsg(LTFS_INFO, 17283I, + (unsigned long long)vol->dp_coh.volume_change_ref, + (unsigned long long)volume_change_ref); + + ret = _ltfs_search_index_wp(recover_symlink, false, &seekpos, vol); + if (ret < 0) + goto out_unlock; + + } else { + ltfsmsg(LTFS_INFO, 17286I, "DP", (unsigned long long)volume_change_ref); + seekpos.partition = ltfs_part_id2num(vol->label->partid_dp, vol); + seekpos.block = vol->dp_coh.set_id; + } } else { if (vollock != PWE_MAM_DP && vollock != PWE_MAM) { /* @@ -1612,13 +1675,37 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec * so this condition wouldn't happen logically. */ ltfsmsg(LTFS_INFO, 17264I, "IP", vl_print); - tape_takedump_drive(vol->device, false); } - seekpos.partition = ltfs_part_id2num(vol->label->partid_ip, vol); - seekpos.block = vol->ip_coh.set_id; - read_ip = true; + + if (volume_change_ref != vol->ip_coh.volume_change_ref) { + /* + * Cannot trust the index info on MAM, search the last indexes + * This would happen when the drive returns an error against acquiring the VCR + * while write error handling. + */ + ltfsmsg(LTFS_INFO, 17283I, + (unsigned long long)vol->dp_coh.volume_change_ref, + (unsigned long long)volume_change_ref); + + /* Index of IP could be corrupted. So set skip flag */ + ret = _ltfs_search_index_wp(recover_symlink, true, &seekpos, vol); + if (ret < 0) + goto out_unlock; + + } else { + ltfsmsg(LTFS_INFO, 17286I, "IP", (unsigned long long)volume_change_ref); + seekpos.partition = ltfs_part_id2num(vol->label->partid_ip, vol); + seekpos.block = vol->ip_coh.set_id; + } } + if (vol->label->part_num2id[seekpos.partition] == vol->label->partid_ip) + read_ip = true; + + ltfsmsg(LTFS_INFO, 17287I, + vol->label->part_num2id[seekpos.partition], + (unsigned long long)seekpos.block); + ret = tape_seek(vol->device, &seekpos); if (ret == -EDEV_EOD_DETECTED) { INTERRUPTED_GOTO(ret, out_unlock); From c3de5ad7fc4eb3e9f6cffa18287f1f919a1139f4 Mon Sep 17 00:00:00 2001 From: piste2750 <32239919+piste2750@users.noreply.github.com> Date: Thu, 19 Aug 2021 20:40:28 +0900 Subject: [PATCH 045/121] Introduce the low level index tool (#309) --- man/Makefile.am | 5 +- man/{ltfs-sde.8 => ltfs.8} | 2 +- man/ltfs_ordered_copy.1 | 2 +- man/ltfsck.8 | 6 +- man/ltfsindextool.8 | 72 +++ man/mkltfs.8 | 6 +- man/sgml/{ltfs-sde.sgml => ltfs.sgml} | 6 +- man/sgml/ltfs_ordered_copy.sgml | 4 +- man/sgml/ltfsck.sgml | 8 +- man/sgml/ltfsindextool.sgml | 159 ++++++ man/sgml/mkltfs.sgml | 8 +- messages/Makefile.am | 1 + messages/bin_ltfsindextool/en.txt | 39 ++ messages/bin_ltfsindextool/en_US.txt | 39 ++ messages/bin_ltfsindextool/root.txt | 109 ++++ src/libltfs/ltfs_error.h | 12 + src/libltfs/xml_reader_libltfs.c | 2 +- src/utils/.gitignore | 1 + src/utils/Makefile.am | 8 +- src/utils/ltfsindextool.c | 775 ++++++++++++++++++++++++++ 20 files changed, 1239 insertions(+), 25 deletions(-) rename man/{ltfs-sde.8 => ltfs.8} (98%) create mode 100644 man/ltfsindextool.8 rename man/sgml/{ltfs-sde.sgml => ltfs.sgml} (98%) create mode 100644 man/sgml/ltfsindextool.sgml create mode 100644 messages/bin_ltfsindextool/en.txt create mode 100644 messages/bin_ltfsindextool/en_US.txt create mode 100644 messages/bin_ltfsindextool/root.txt create mode 100644 src/utils/ltfsindextool.c diff --git a/man/Makefile.am b/man/Makefile.am index f18e081d..f6eec413 100644 --- a/man/Makefile.am +++ b/man/Makefile.am @@ -1,4 +1,4 @@ -man_MANS = mkltfs.8 ltfsck.8 ltfs-sde.8 ltfs_ordered_copy.1 +man_MANS = mkltfs.8 ltfsck.8 ltfs.8 ltfs_ordered_copy.1 ltfsindextool.8 man-clean: rm $(man_MANS) @@ -6,6 +6,7 @@ man-clean: man-rebuild: docbook2man sgml/mkltfs.sgml docbook2man sgml/ltfsck.sgml - docbook2man sgml/ltfs-sde.sgml + docbook2man sgml/ltfs.sgml docbook2man sgml/ltfs_ordered_copy.sgml + docbook2man sgml/ltfsindextool.sgml rm -f manpage.* diff --git a/man/ltfs-sde.8 b/man/ltfs.8 similarity index 98% rename from man/ltfs-sde.8 rename to man/ltfs.8 index 3ac31bf9..e7e39331 100644 --- a/man/ltfs-sde.8 +++ b/man/ltfs.8 @@ -1,5 +1,5 @@ .\" auto-generated by docbook2man-spec from docbook-utils package -.TH "LTFS-SDE" "8" "16 July 2020" "IBM Spectrum Archive" "IBM Spectrum Archive Command Reference" +.TH "LTFS" "8" "19 August 2021" "LTFS" "LTFS Command Reference" .SH NAME ltfs \- File system based on a linear tape drive .SH SYNOPSIS diff --git a/man/ltfs_ordered_copy.1 b/man/ltfs_ordered_copy.1 index 0420c469..f5206629 100644 --- a/man/ltfs_ordered_copy.1 +++ b/man/ltfs_ordered_copy.1 @@ -1,5 +1,5 @@ .\" auto-generated by docbook2man-spec from docbook-utils package -.TH "LTFS_ORDERED_COPY" "1" "16 July 2020" "IBM Spectrum Archive" "IBM Spectrum Archive Command Reference" +.TH "LTFS_ORDERED_COPY" "1" "19 August 2021" "LTFS" "LTFS Command Reference" .SH NAME ltfs_ordered_copy \- Copy files from source to destination with LTFS order optimization .SH SYNOPSIS diff --git a/man/ltfsck.8 b/man/ltfsck.8 index 3961e343..378e6990 100644 --- a/man/ltfsck.8 +++ b/man/ltfsck.8 @@ -1,5 +1,5 @@ .\" auto-generated by docbook2man-spec from docbook-utils package -.TH "LTFSCK" "8" "16 July 2020" "IBM Spectrum Archive" "IBM Spectrum Archive Command Reference" +.TH "LTFSCK" "8" "19 August 2021" "LTFS" "LTFS Command Reference" .SH NAME ltfsck \- Recover and rollback utility for LTFS formatted tape .SH SYNOPSIS @@ -11,7 +11,7 @@ On Linux, \fIdevice_name\fR is like .SH "DESCRIPTION" .PP \fBltfsck\fR is a program to recover an inconsistent -LTFS formatted medium and roll back utility of the IBM Spectrum Archive. +LTFS formatted medium and roll back utility of the LTFS. .SH "OPTIONS" .PP These programs follow the usual GNU command line syntax, @@ -98,4 +98,4 @@ Capture index information to the current directory List the rollback points of the cartridge that has no EOD .SH "SEE ALSO" .PP -ltfs-sde(8), mkltfs(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). +ltfs(8), mkltfs(8), ltfsindextool(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). diff --git a/man/ltfsindextool.8 b/man/ltfsindextool.8 new file mode 100644 index 00000000..dd932c29 --- /dev/null +++ b/man/ltfsindextool.8 @@ -0,0 +1,72 @@ +.\" auto-generated by docbook2man-spec from docbook-utils package +.TH "LTFSINDEXTOOL" "8" "19 August 2021" "LTFS" "LTFS Command Reference" +.SH NAME +ltfsindextool \- Low level index checking tool for LTFS +.SH SYNOPSIS +.sp +\fBltfsindextool\fR [ \fB-d \fIname\fB\fR ] [ \fB-p \fIpart_num\fB\fR ] [ \fB-s \fIblock\fB\fR ] [ \fB-b \fInum\fB\fR ] [ \fB-i \fIfile\fB\fR ] [ \fB-e \fIname\fB\fR ] [ \fB-q\fR ] [ \fB-t\fR ] [ \fB-V\fR ] [ \fB-h\fR ] [ \fB-p\fR ] [ \fB\fIfilename\fB\fR ] +.SH "DESCRIPTION" +.PP +\fBltfsindextool\fR is a low level index checking tool. +.PP +There are 2 features. One is captureing indexes on a tape, the other is checking captured index from a file. +The command runs as index checking mode when filename is specified. It runs as index capturing mode when +-d option is specified. It runs as index checking mode when both filename and -d option are specified. +-p, -s --output-dir, -b is available only when it runs under index captureing mode. They would be ignored +when it runs under index checking mode. +.SH "OPTIONS" +.PP +These programs follow the usual GNU command line syntax, +with long options starting with two dashes ('-'). A summary of +options is included below. For a complete description, see the +\fBInfo\fR files. +.TP +\fB-d, --device=\fIname\fB\fR +Tape device name to capture indexes. On Linux, \fIname\fR is like +\&'/dev/IBMtape0', on OSX, \fIname\fR is like '0'. +.TP +\fB-p, --partition=\fIpart_num\fB\fR +Partition to capture indexes. Shall be 0 or 1. Capture indexes on both partitions +.TP +\fB-s, --start-pos=\fIblock\fB\fR +Block number to start capturing indexes +.TP +\fB--output-dir=\fIdir\fB\fR +Directory to store captured indexes +.TP +\fB-b, --blocksize=\fInum\fB\fR +Specify the LTFS record size +.TP +\fB-i, --config=\fIname\fB\fR +Use the specified configuration file +.TP +\fB-e, --backend=\fIname\fB\fR +Use the specified tape device backend +.TP +\fB--kmi-backend=\fIname\fB\fR +Use the specified key manager interface backend (default: none) +.TP +\fB-q, --quiet\fR +Suppress progress information and general messages +.TP +\fB-t, --trace\fR +Enable function call tracing +.TP +\fB-V, --version\fR +Version information +.TP +\fB-h, --help\fR +Show help information +.SS "USAGE EXAMPLE" +.sp +.nf + ltfsindextool -d /dev/sg10 + ltfsindextool -d /dev/sg10 -p 1 --output-dir=/foo + ltfsindextool -d ltfs-index-1-35.xml + +.sp +.fi +.PP +.SH "SEE ALSO" +.PP +ltfs(8), mkltfs(8), ltfsck(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). diff --git a/man/mkltfs.8 b/man/mkltfs.8 index 37e59e28..ed629fe2 100644 --- a/man/mkltfs.8 +++ b/man/mkltfs.8 @@ -1,5 +1,5 @@ .\" auto-generated by docbook2man-spec from docbook-utils package -.TH "MKLTFS" "8" "16 July 2020" "IBM Spectrum Archive" "IBM Spectrum Archive Command Reference" +.TH "MKLTFS" "8" "19 August 2021" "LTFS" "LTFS Command Reference" .SH NAME mkltfs \- Format a tape in the drive to LTFS format .SH SYNOPSIS @@ -8,7 +8,7 @@ mkltfs \- Format a tape in the drive to LTFS format .SH "DESCRIPTION" .PP \fBmkltfs\fR is a program to format a media for use with -the IBM Spectrum Archive. +the LTFS. .SH "OPTIONS" .PP These programs follow the usual GNU command line syntax, @@ -114,4 +114,4 @@ Unformat the medium and erase any data on the tape by overwriting special data p This operation takes over 3 hours. Once you start, you cannot interrupt it. .SH "SEE ALSO" .PP -ltfs-sde(8), ltfsck(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). +ltfs(8), ltfsck(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). diff --git a/man/sgml/ltfs-sde.sgml b/man/sgml/ltfs.sgml similarity index 98% rename from man/sgml/ltfs-sde.sgml rename to man/sgml/ltfs.sgml index 9fc854b6..69d34b61 100644 --- a/man/sgml/ltfs-sde.sgml +++ b/man/sgml/ltfs.sgml @@ -1,10 +1,10 @@ + GNU"> ]> - IBM Spectrum Archive Command Reference + LTFS Command Reference @@ -12,7 +12,7 @@ &dhcommand; 8 - IBM Spectrum Archive + LTFS diff --git a/man/sgml/ltfs_ordered_copy.sgml b/man/sgml/ltfs_ordered_copy.sgml index e56d73c8..fed9994f 100644 --- a/man/sgml/ltfs_ordered_copy.sgml +++ b/man/sgml/ltfs_ordered_copy.sgml @@ -4,7 +4,7 @@ ]> - IBM Spectrum Archive Command Reference + LTFS Command Reference @@ -12,7 +12,7 @@ &dhcommand; 1 - IBM Spectrum Archive + LTFS diff --git a/man/sgml/ltfsck.sgml b/man/sgml/ltfsck.sgml index ac28a61c..4a8d023f 100644 --- a/man/sgml/ltfsck.sgml +++ b/man/sgml/ltfsck.sgml @@ -4,7 +4,7 @@ M - IBM Spectrum Archive Command Reference + LTFS Command Reference @@ -12,7 +12,7 @@ M&dhcommand; 8 - IBM Spectrum Archive + LTFS @@ -51,7 +51,7 @@ MDESCRIPTION &dhcommand; is a program to recover an inconsistent - LTFS formatted medium and roll back utility of the IBM Spectrum Archive. + LTFS formatted medium and roll back utility of the LTFS. @@ -228,7 +228,7 @@ M SEE ALSO - ltfs-sde(8), mkltfs(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). + ltfs(8), mkltfs(8), ltfsindextool(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). diff --git a/man/sgml/ltfsindextool.sgml b/man/sgml/ltfsindextool.sgml new file mode 100644 index 00000000..d4530105 --- /dev/null +++ b/man/sgml/ltfsindextool.sgml @@ -0,0 +1,159 @@ + + GNU"> + ]> + + + LTFS Command Reference + + + + + &dhcommand; + + 8 + LTFS + + + + &dhcommand; + Low level index checking tool for LTFS + + + + + &dhcommand; + + + + + + + + + + + + filename + + + + + DESCRIPTION + + &dhcommand; is a low level index checking tool. + + + There are 2 features. One is captureing indexes on a tape, the other is checking captured index from a file. + The command runs as index checking mode when filename is specified. It runs as index capturing mode when + -d option is specified. It runs as index checking mode when both filename and -d option are specified. + -p, -s --output-dir, -b is available only when it runs under index captureing mode. They would be ignored + when it runs under index checking mode. + + + + + OPTIONS + + These programs follow the usual &gnu; command line syntax, + with long options starting with two dashes ('-'). A summary of + options is included below. For a complete description, see the + Info files. + + + + + + + + Tape device name to capture indexes. On Linux, name is like + '/dev/IBMtape0', on OSX, name is like '0'. + + + + + + + Partition to capture indexes. Shall be 0 or 1. Capture indexes on both partitions + + + + + + Block number to start capturing indexes + + + + + + Directory to store captured indexes + + + + + + Specify the LTFS record size + + + + + + Use the specified configuration file + + + + + + Use the specified tape device backend + + + + + + Use the specified key manager interface backend (default: none) + + + + + + Suppress progress information and general messages + + + + + + Enable function call tracing + + + + + + Version information + + + + + + Show help information + + + + + + USAGE EXAMPLE + + ltfsindextool -d /dev/sg10 + ltfsindextool -d /dev/sg10 -p 1 --output-dir=/foo + ltfsindextool -d ltfs-index-1-35.xml + + + + + + SEE ALSO + ltfs(8), mkltfs(8), ltfsck(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). + + + + + diff --git a/man/sgml/mkltfs.sgml b/man/sgml/mkltfs.sgml index f7d0aa31..d0ed00ed 100644 --- a/man/sgml/mkltfs.sgml +++ b/man/sgml/mkltfs.sgml @@ -4,7 +4,7 @@ ]> - IBM Spectrum Archive Command Reference + LTFS Command Reference @@ -12,7 +12,7 @@ &dhcommand; 8 - IBM Spectrum Archive + LTFS @@ -41,7 +41,7 @@ DESCRIPTION &dhcommand; is a program to format a media for use with - the IBM Spectrum Archive. + the LTFS. @@ -233,7 +233,7 @@ SEE ALSO - ltfs-sde(8), ltfsck(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). + ltfs(8), ltfsck(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). diff --git a/messages/Makefile.am b/messages/Makefile.am index 3d712e00..26bc61dc 100644 --- a/messages/Makefile.am +++ b/messages/Makefile.am @@ -49,6 +49,7 @@ RESOURCES = \ libbin_mkltfs_dat.a \ libbin_ltfsck_dat.a \ + libbin_ltfsindextool_dat.a \ libbin_ltfs_dat.a \ libkmi_simple_dat.a \ libkmi_flatfile_dat.a \ diff --git a/messages/bin_ltfsindextool/en.txt b/messages/bin_ltfsindextool/en.txt new file mode 100644 index 00000000..44064bb4 --- /dev/null +++ b/messages/bin_ltfsindextool/en.txt @@ -0,0 +1,39 @@ +// +// +// OO_Copyright_BEGIN +// +// +// Copyright 2010, 2020 IBM Corp. All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// 3. Neither the name of the copyright holder nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS'' +// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE +// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +// POSSIBILITY OF SUCH DAMAGE. +// +// +// OO_Copyright_END +// + +en:table { + // This resource intentionally left blank. +} + diff --git a/messages/bin_ltfsindextool/en_US.txt b/messages/bin_ltfsindextool/en_US.txt new file mode 100644 index 00000000..ffd90dd2 --- /dev/null +++ b/messages/bin_ltfsindextool/en_US.txt @@ -0,0 +1,39 @@ +// +// +// OO_Copyright_BEGIN +// +// +// Copyright 2010, 2020 IBM Corp. All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// 3. Neither the name of the copyright holder nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS'' +// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE +// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +// POSSIBILITY OF SUCH DAMAGE. +// +// +// OO_Copyright_END +// + +en_US:table { + // This resource intentionally left blank. +} + diff --git a/messages/bin_ltfsindextool/root.txt b/messages/bin_ltfsindextool/root.txt new file mode 100644 index 00000000..b84dc906 --- /dev/null +++ b/messages/bin_ltfsindextool/root.txt @@ -0,0 +1,109 @@ +// +// +// OO_Copyright_BEGIN +// +// +// Copyright 2010, 2020 IBM Corp. All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions +// are met: +// 1. Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// 2. Redistributions in binary form must reproduce the above copyright +// notice, this list of conditions and the following disclaimer in the +// documentation and/or other materials provided with the distribution. +// 3. Neither the name of the copyright holder nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS'' +// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE +// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +// POSSIBILITY OF SUCH DAMAGE. +// +// +// OO_Copyright_END +// + +// Messages for ltfsindextool. +root:table { + messages:table { + start_id:int { 19500 } + end_id: int { 19989 } + 19500I:string { "Starting ltfsindextool, %s version %s, log level %d." } + 19501E:string { "Cannot allocate LTFS volume structure." } + 19502I:string { "%s." } + 19503I:string { "GCC version is %s." } + 19504I:string { "Capture all indexes from both partitions." } + 19505I:string { "Capture all indexes from (%u, %llu)." } + 19506D:string { "Opening the device." } + 19507D:string { "Device opened." } + 19508E:string { "Cannot open backend \'%s\'." } + 19509E:string { "Cannot open key manager interface backend \'%s\'." } + 19510E:string { "Cannot open device \'%s\' (%d)." } + 19511E:string { "Could not initialize the key manager interface plug-in. \'%s\' (%d)." } + 19512E:string { "Key manager interface backend option parsing failed." } + 19513E:string { "Tape backend option parsing failed." } + 19514E:string { "Unknown option '%s %s'." } + 19515E:string { "Cannot set up tape device." } + 19516E:string { "Cannot allocate the read buffer." } + 19517E:string { "Cannot seek to the start position (%u, %llu, %d)." } + 19518E:string { "Cannot get the current position (%d)." } + 19519E:string { "Cannot read a block (%u, %llu, %ld)." } + 19520D:string { "Closing the device." } + 19521W:string { "Cannot unload backend." } + 19522D:string { "Device closed." } + 19523I:string { "Failed to capture index (%d)." } + 19524I:string { "Captured indexes successfully." } + 19525D:string { "Validating command line options." } + 19526E:string { "File name to check or device name to capture must be specified." } + 19527D:string { "Command line options are valid." } + 19528W:string { "Cannot unload key manager interface backend." } + 19529I:string { "Reading an index from (%u, %llu)." } + 19530I:string { "Block in (%u, %llu) does not look an index, seek to next position (%d)." } + 19531E:string { "Cannot seek to the next position (%u, %llu, %d)." } + 19532E:string { "Cannot allocate the file name." } + 19533E:string { "Cannot open the file %s (%d)." } + 19534I:string { "Reached to EOD (%u, %llu)." } + 19535I:string { "Reached to EOD but cannot get the position (%d)." } + 19536E:string { "Cannot write a block (%ld, %d)." } + 19537I:string { "Detected the end of the index (%u, %llu)." } + 19538E:string { "Detected the EOD in the middle of index (%u, %llu)." } + 19539I:string { "Wrote an index, length is %llu." } + 19540E:string { "Partition number must be 0 or 1." } + 19541E:string { "Operation mode is wrong." } + 19542I:string { "Launched by \"%s\"." } + 19543I:string { "Checking the index file \"%s\"." } + 19544I:string { "Checked the index successfully." } + 19545E:string { "Failed to check the index (%d)." } + 19546I:string { "%s version %s." } + 19547I:string { "Creating an index file: %s" } + 19548E:string { "Start position shall be 5 or larger (%llu)." } + + // Help messages + 19900I:string { "Usage: %s [options] [filename]" } + 19901I:string { "Available options are:" } + 19902I:string { " -d, --device= Tape device (Capture index to specified file when this option is specified. Otherwise run check process with specified file)" } + 19903I:string { " -p, --partition= Partition number 0 or 1, capture both partitions if not specified" } + 19904I:string { " -s, --start-pos= Block number to start capturing (default: %d)" } + 19905I:string { " --output-dir= Directory to store captured indexes (default: \'%s\')" } + 19906I:string { " -b, --blocksize= Specify the LTFS record size (default: %d)" } + 19907I:string { " -i, --config= Use the specified configuration file (default: %s)" } + 19908I:string { " -e, --backend= Use the specified tape device backend (default: %s)" } + 19909I:string { " --kmi-backend= Use the specified key manager interface backend (default: %s)" } + 19910I:string { " -q, --quiet Suppress progress information and general messages" } + 19911I:string { " -t, --trace Enable function call tracing" } + 19912I:string { " -V, --version Version information" } + 19913I:string { " -h, --help This help" } + 19914I:string { "Usage example:" } + 19915I:string { " %s --device=%s -p=%d" } + } +} diff --git a/src/libltfs/ltfs_error.h b/src/libltfs/ltfs_error.h index 433ec15b..acf7b20d 100644 --- a/src/libltfs/ltfs_error.h +++ b/src/libltfs/ltfs_error.h @@ -517,4 +517,16 @@ #define MKLTFS_USAGE_SYNTAX_ERROR PROG_USAGE_SYNTAX_ERROR /* Wrong argument */ #define MKLTFS_CANCELED_BY_USER PROG_CANCELED_BY_USER /* Canceled by user */ +/* Status code for ltfsindextool + * The maximum return code from the program is 0xFF. + */ +#define INDEXTOOL_NO_ERRORS PROG_NO_ERRORS /* No error and the cartridge is not modified */ +#define INDEXTOOL_CORRECTED PROG_TREAT_SUCCESS /* Recover correctly, the cartridge is modified */ +#define INDEXTOOL_REBOOT_REQUIRED PROG_REBOOT_REQUIRED /* Reboot required */ +#define INDEXTOOL_UNCORRECTED PROG_UNCORRECTED /* Cannot recover, the cartridge is modified */ +#define INDEXTOOL_OPERATIONAL_ERROR PROG_OPERATIONAL_ERROR /* Get device error while processing, the cartridge may be modified */ +#define INDEXTOOL_USAGE_SYNTAX_ERROR PROG_USAGE_SYNTAX_ERROR /* Wrong argument */ +#define INDEXTOOL_CANCELED_BY_USER PROG_CANCELED_BY_USER /* Canceled by user */ +#define INDEXTOOL_SHARED_LIB_ERROR PROG_SHARED_LIB_ERROR /* Library error */ + #endif /* __ltfs_error_h__ */ diff --git a/src/libltfs/xml_reader_libltfs.c b/src/libltfs/xml_reader_libltfs.c index 92ba852b..bec3d56a 100644 --- a/src/libltfs/xml_reader_libltfs.c +++ b/src/libltfs/xml_reader_libltfs.c @@ -1838,7 +1838,7 @@ int xml_schema_from_file(const char *filename, struct ltfs_index *idx, struct lt reader = xmlReaderForFile(filename, NULL, XML_PARSE_NOERROR | XML_PARSE_NOWARNING | XML_PARSE_HUGE); if (! reader) { ltfsmsg(LTFS_ERR, 17011E, filename); - return -1; + return -LTFS_FILE_ERR; } /* Workaround for old libxml2 version on OS X 10.5: the method used to preserve diff --git a/src/utils/.gitignore b/src/utils/.gitignore index 37cf58f7..69d22ee6 100644 --- a/src/utils/.gitignore +++ b/src/utils/.gitignore @@ -1,2 +1,3 @@ ltfsck mkltfs +ltfsindextool diff --git a/src/utils/Makefile.am b/src/utils/Makefile.am index 786b8670..1a7a7e3c 100644 --- a/src/utils/Makefile.am +++ b/src/utils/Makefile.am @@ -33,7 +33,7 @@ # OO_Copyright_END # -bin_PROGRAMS = mkltfs ltfsck +bin_PROGRAMS = mkltfs ltfsck ltfsindextool nobase_bin_SCRIPTS = ltfs_ordered_copy @@ -50,3 +50,9 @@ ltfsck_DEPENDENCIES = ../libltfs/libltfs.la ../../messages/libbin_ltfsck_dat.a ltfsck_LDADD = ../libltfs/libltfs.la ltfsck_LDFLAGS = @AM_LDFLAGS@ -L../../messages -lbin_ltfsck_dat ltfsck_CPPFLAGS = @AM_CPPFLAGS@ -I .. -fPIC + +ltfsindextool_SOURCES = ltfsindextool.c +ltfsindextool_DEPENDENCIES = ../libltfs/libltfs.la ../../messages/libbin_ltfsindextool_dat.a ../../messages/libbin_mkltfs_dat.a +ltfsindextool_LDADD = ../libltfs/libltfs.la +ltfsindextool_LDFLAGS = @AM_LDFLAGS@ -L../../messages -lbin_ltfsindextool_dat +ltfsindextool_CPPFLAGS = @AM_CPPFLAGS@ -I .. -fPIC diff --git a/src/utils/ltfsindextool.c b/src/utils/ltfsindextool.c new file mode 100644 index 00000000..c4d47a64 --- /dev/null +++ b/src/utils/ltfsindextool.c @@ -0,0 +1,775 @@ +/* +** +** OO_Copyright_BEGIN +** +** +** Copyright 2010, 2020 IBM Corp. All rights reserved. +** +** Redistribution and use in source and binary forms, with or without +** modification, are permitted provided that the following conditions +** are met: +** 1. Redistributions of source code must retain the above copyright +** notice, this list of conditions and the following disclaimer. +** 2. Redistributions in binary form must reproduce the above copyright +** notice, this list of conditions and the following disclaimer in the +** documentation and/or other materials provided with the distribution. +** 3. Neither the name of the copyright holder nor the names of its +** contributors may be used to endorse or promote products derived from +** this software without specific prior written permission. +** +** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS'' +** AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +** IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +** ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE +** LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +** CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +** SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +** INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +** CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +** ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +** POSSIBILITY OF SUCH DAMAGE. +** +** +** OO_Copyright_END +** +************************************************************************************* +** +** COMPONENT NAME: IBM Linear Tape File System +** +** FILE NAME: utils/ltfsindextool.c +** +** DESCRIPTION: The low level index tool +** +** AUTHORS: Atsushi Abe +** piste.2750@gmail.com +** +************************************************************************************* +*/ + +#ifdef mingw_PLATFORM +#include "libltfs/arch/win/win_util.h" +#else +#include +#endif /* mingw_PLATFORM */ + +#include +#include "libltfs/ltfs_fuse_version.h" +#include + +#include "libltfs/ltfs.h" +#include "libltfs/xml_libltfs.h" +#include "ltfs_copyright.h" +#include "libltfs/plugin.h" +#include "libltfs/kmi.h" +#include "libltfs/tape.h" + +volatile char *copyright = LTFS_COPYRIGHT_0"\n"LTFS_COPYRIGHT_1"\n"LTFS_COPYRIGHT_2"\n" \ + LTFS_COPYRIGHT_3"\n"LTFS_COPYRIGHT_4"\n"LTFS_COPYRIGHT_5"\n"; + +#ifdef __APPLE__ +#include "libltfs/arch/osx/osx_string.h" +#endif + +#ifdef mingw_PLATFORM +char *bin_ltfsindextool_dat; +#else +extern char bin_ltfsindextool_dat[]; +#endif + +typedef enum { + OP_CHECK, + OP_CAPTURE, +} operation_mode_t; + +struct indextool_opts { + operation_mode_t mode; /**< Operation mode */ + char *filename; /**< Index filename to check in check mode */ + char *devname; /**< Device name to capture index in capture mode */ + int partition; /**< partition to operate */ + uint64_t start_pos; /**< start position */ + char *out_dir; /**< Output dir for captured indexes */ + unsigned long blocksize; /**< Nominal tape block size */ + struct config_file *config; /**< Configuration data read from the global LTFS config file */ + char *backend_path; /**< Path to tape backend shared library */ + char *kmi_backend_name; /**< Name or path to the key manager interface backend library */ + bool quiet; /**< Quiet mode indicator */ + bool trace; /**< Debug mode indicator */ + bool syslogtrace; /**< Generate debug output to stderr and syslog*/ +}; + +/* Command line options */ +#define PART_BOTH (-1) +#define START_POS (5) +#define OUTPUT_DIR "." +#define KEY_MAX_OFFSET (0x30) + +static const char *short_options = "i:e:d:p:s:o:b:qthV"; +static struct option long_options[] = { + {"config", 1, 0, 'i'}, + {"backend", 1, 0, 'e'}, + {"device", 1, 0, 'd'}, + {"partition", 1, 0, 'p'}, + {"start-pos", 1, 0, 's'}, + {"output-dir", 1, 0, '^'}, + {"blocksize", 1, 0, 'b'}, + {"kmi-backend", 1, 0, '-'}, + {"quiet", 0, 0, 'q'}, + {"trace", 0, 0, 't'}, + {"syslogtrace", 0, 0, '!'}, + {"help", 0, 0, 'h'}, + {"version", 0, 0, 'V'}, + {0, 0, 0, 0} +}; + +/* Private functions */ +static inline int _open_output_file(tape_partition_t part, + tape_block_t start_pos, + char *base_path) +{ + int ret; + char *fname = NULL; + + ret = asprintf(&fname, "%s/ltfs-index-%u-%llu.xml", base_path, + (unsigned int) part, (unsigned long long)start_pos); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19532E); + return -1; + } + + ltfsmsg(LTFS_INFO, 19547I, fname); + ret = open(fname, O_WRONLY | O_CREAT | O_TRUNC, 0666); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19533E, fname, errno); + } + + free(fname); + + return ret; +} + +static inline void _close_output_file(int fd) +{ + fsync(fd); + close(fd); +} + +/** Capture indexes on the partition. + * + */ +static int ltfs_capture_index_raw(tape_partition_t part, + tape_block_t start_pos, + int blocksize, + char *base_path, + struct ltfs_volume *vol) +{ + int ret = 0, fd = -1; + ssize_t nread, nwrite, index_len = 0; + struct tc_position pos; + char *buf = NULL, *key = NULL, check_buf[KEY_MAX_OFFSET + 1]; + + pos.partition = part; + pos.block = start_pos; + + buf = malloc(blocksize); + if (!buf) { + ltfsmsg(LTFS_ERR, 19516E); + return -LTFS_NO_MEMORY; + } + + ret = tape_seek(vol->device, &pos); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19517E, (unsigned int)part, (unsigned long long)start_pos, ret); + return ret; + } + + while (ret == 0) { + index_len = 0; + + ret = tape_get_position(vol->device, &pos); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19518E, ret); + break; + } + + nread = tape_read(vol->device, buf, blocksize, true, vol->kmi_handle); + if (nread < 0) { + ltfsmsg(LTFS_ERR, 19519E, + (unsigned int)pos.partition, (unsigned long long)pos.block, nread); + ret = nread; + break; + } + pos.block++; + + memset(check_buf, 0x00, KEY_MAX_OFFSET + 1); + strncpy(check_buf, buf, KEY_MAX_OFFSET); + key = strstr(check_buf, "device, buf, blocksize, true, vol->kmi_handle); + if (nread == -EDEV_EOD_DETECTED) { + ret = nread; + ltfsmsg(LTFS_ERR, 19538E, + (unsigned int)pos.partition, + (unsigned long long)(pos.block)); + ltfsmsg(LTFS_INFO, 19539I, (unsigned long long)index_len); + break; + } + + if (nread > 0) { + /* Write a block to the file */ + nwrite = write(fd, buf, nread); + if (nwrite == nread) { + index_len += nread; + } else { + ltfsmsg(LTFS_ERR, 19536E, nwrite, errno); + _close_output_file(fd); + return -LTFS_CACHE_IO; + } + pos.block++; + } else if (!nread) { + /* Detect a FM (the end of the index), do nothing */ + ltfsmsg(LTFS_INFO, 19537I, + (unsigned int)pos.partition, + (unsigned long long)(pos.block)); + ltfsmsg(LTFS_INFO, 19539I, (unsigned long long)index_len); + break; + } else { + ltfsmsg(LTFS_ERR, 19519E, + (unsigned int)pos.partition, (unsigned long long)pos.block, nread); + ret = nread; + break; + } + } + + _close_output_file(fd); + } else { + /* seek to next FM */ + if (key) + ltfsmsg(LTFS_INFO, 19530I, + (unsigned int)pos.partition, + (unsigned long long)(pos.block - 1), + (int)(key - buf)); + else + ltfsmsg(LTFS_INFO, 19530I, + (unsigned int)pos.partition, + (unsigned long long)(pos.block - 1), + (int)0); + + /* Do nothig at (nread == 0) because tape hits a FM */ + if (nread > 0) { + ret = tape_spacefm(vol->device, 1); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19531E, (unsigned int)part, (unsigned long long)start_pos, ret); + break; + } + } + } + } + + if (ret == -EDEV_EOD_DETECTED) { + ret = tape_get_position(vol->device, &pos); + if (!ret) { + ltfsmsg(LTFS_INFO, 19534I,(unsigned int)pos.partition, (unsigned long long)pos.block); + } else { + ltfsmsg(LTFS_INFO, 19535I, ret); + } + ret = 0; + } + + if (buf) + free(buf); + + return ret; +} + +static int _capture(struct indextool_opts *opt, struct ltfs_volume *vol) +{ + int r, ret = 0; + tape_partition_t p; + + if (opt->partition == PART_BOTH) { + ltfsmsg(LTFS_INFO, 19504I); + for (p = 0; p < 2; p++) { + r = ltfs_capture_index_raw(p, 5, opt->blocksize, opt->out_dir, vol); + if (!ret) + ret = r; + } + } else { + ltfsmsg(LTFS_INFO, 19505I, (unsigned int)opt->partition, (unsigned long long)opt->start_pos); + ret = ltfs_capture_index_raw(opt->partition, opt->start_pos, opt->blocksize, opt->out_dir, vol); + } + + return ret; +} + +static int _indextool_validate_options(char *prg_name, struct indextool_opts *opt) +{ + ltfsmsg(LTFS_DEBUG, 19525D); + + /* Validate filename and devname and decide a operation mode*/ + if (opt->filename) { + opt->mode = OP_CHECK; + } else if (opt->devname) { + opt->mode = OP_CAPTURE; + } else { + ltfsmsg(LTFS_ERR, 19526E); + return 1; + } + + /* Validate partition */ + if (opt->partition != PART_BOTH && opt->partition != 0 && opt->partition != 1) { + ltfsmsg(LTFS_ERR, 19540E); + return 1; + } + + /* Validate start position */ + if (opt->start_pos < START_POS) { + ltfsmsg(LTFS_ERR, 19548E, (unsigned long long)opt->start_pos); + return 1; + } + + ltfsmsg(LTFS_DEBUG, 19527D); + return 0; +} + +static int check_index(struct ltfs_volume *vol, struct indextool_opts *opt, void *args) +{ + int ret = 0; + + ltfsmsg(LTFS_INFO, 19543I, opt->filename); + + vol->label->blocksize = opt->blocksize; + ret = xml_schema_from_file(opt->filename, vol->index, vol); + + if (!ret) { + ltfsmsg(LTFS_INFO, 19544I); + } else { + ltfsmsg(LTFS_ERR, 19545E, ret); + } + + return ret; +} + +static int capture_index(struct ltfs_volume *vol, struct indextool_opts *opt, void *args) +{ + int ret = INDEXTOOL_OPERATIONAL_ERROR; + struct libltfs_plugin backend; /* tape driver backend */ + struct libltfs_plugin kmi; /* key manager interface backend */ + + /* load the backend, open the tape device, and load a tape */ + ltfsmsg(LTFS_DEBUG, 19506D); + ret = plugin_load(&backend, "tape", opt->backend_path, opt->config); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19508E, opt->backend_path); + return INDEXTOOL_OPERATIONAL_ERROR; + } + if (opt->kmi_backend_name) { + ret = plugin_load(&kmi, "kmi", opt->kmi_backend_name, opt->config); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19509E, opt->kmi_backend_name); + return INDEXTOOL_OPERATIONAL_ERROR; + } + } + ret = ltfs_device_open(opt->devname, backend.ops, vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19510E, opt->devname, ret); + ret = INDEXTOOL_OPERATIONAL_ERROR; + goto out_unload_backend; + } + ret = ltfs_parse_tape_backend_opts(args, vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19513E); + goto out_unload_backend; + } + if (opt->kmi_backend_name) { + ret = kmi_init(&kmi, vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19511E, opt->devname, ret); + goto out_unload_backend; + } + + ret = ltfs_parse_kmi_backend_opts(args, vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19512E); + goto out_unload_backend; + } + + ret = tape_clear_key(vol->device, vol->kmi_handle); + if (ret < 0) + goto out_unload_backend; + } + + { + int i = 0; + struct fuse_args *a = args; + + for (i = 0; i < a->argc && a->argv[i]; ++i) { + if (!strcmp(a->argv[i], "-o")) { + ltfsmsg(LTFS_ERR, 19514E, a->argv[i], a->argv[i + 1] ? a->argv[i + 1] : ""); + ret = INDEXTOOL_USAGE_SYNTAX_ERROR; + goto out_unload_backend; + } + } + } + + ltfs_load_tape(vol); + ret = ltfs_wait_device_ready(vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19515E); + ret = INDEXTOOL_OPERATIONAL_ERROR; + goto out_close; + } + + vol->append_only_mode = false; + vol->set_pew = false; + ret = ltfs_setup_device(vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19515E); + ret = INDEXTOOL_OPERATIONAL_ERROR; + goto out_close; + } + ltfsmsg(LTFS_DEBUG, 19507D); + + /* Capture_index */ + ret = _capture(opt, vol); + + /* close the tape device and unload the backend */ + ltfsmsg(LTFS_DEBUG, 19520D); + +out_close: + ltfs_device_close(vol); + ltfs_volume_free(&vol); + ltfs_unset_signal_handlers(); + if (ret == INDEXTOOL_NO_ERRORS) + ltfsmsg(LTFS_DEBUG, 19522D); +out_unload_backend: + if (ret == INDEXTOOL_NO_ERRORS) { + ret = plugin_unload(&backend); + if (ret < 0) + ltfsmsg(LTFS_WARN, 19521W); + if (opt->kmi_backend_name) { + ret = plugin_unload(&kmi); + if (ret < 0) + ltfsmsg(LTFS_WARN, 19528W); + } + ret = INDEXTOOL_NO_ERRORS; + } else { + plugin_unload(&backend); + if (opt->kmi_backend_name) + plugin_unload(&kmi); + } + + if (ret == INDEXTOOL_NO_ERRORS) + ltfsmsg(LTFS_INFO, 19524I); + else + ltfsmsg(LTFS_INFO, 19523I, ret); + + return ret; +} + +static void show_usage(char *appname, struct config_file *config, bool full) +{ + struct libltfs_plugin backend; + const char *default_backend; + char *devname = NULL; + + default_backend = config_file_get_default_plugin("tape", config); + if (default_backend && plugin_load(&backend, "tape", default_backend, config) == 0) { + devname = strdup(ltfs_default_device_name(backend.ops)); + plugin_unload(&backend); + } + + if (! devname) + devname = strdup(""); + + fprintf(stderr, "\n"); + ltfsresult(19900I, appname); /* Usage: %s */ + fprintf(stderr, "\n"); + ltfsresult(19901I); /* Available options are: */ + ltfsresult(19902I); /* -d, --device= */ + ltfsresult(19903I); /* -p, --partition=<0|1> */ + ltfsresult(19904I, START_POS); /* -s, --start-pos */ + ltfsresult(19905I, OUTPUT_DIR); /* -output-dir */ + ltfsresult(19906I, LTFS_DEFAULT_BLOCKSIZE); /* -b, --blocksize */ + ltfsresult(19907I, LTFS_CONFIG_FILE); /* -i, --config= */ + ltfsresult(19908I, default_backend); /* -e, --backend */ + ltfsresult(19909I, config_file_get_default_plugin("kmi", config)); /* --kmi-backend */ + ltfsresult(19910I); /* -q, --quiet */ + ltfsresult(19911I); /* -t, --trace */ + ltfsresult(19912I); /* -V, --version */ + ltfsresult(19913I); /* -h, --help */ + fprintf(stderr, "\n"); + plugin_usage(appname, "driver", config); + fprintf(stderr, "\n"); + plugin_usage(appname, "kmi", config); + fprintf(stderr, "\n"); + ltfsresult(19914I); /* Usage example: */ + ltfsresult(19915I, appname, devname, 0); + free(devname); +} + +/* Main routine */ +int main(int argc, char **argv) +{ + struct ltfs_volume *vol; + struct indextool_opts opt; + int ret, log_level, syslog_level, i, cmd_args_len; + char *lang, *cmd_args; + const char *config_file = NULL; + void *message_handle; + + int fuse_argc = argc; + char **fuse_argv = calloc(fuse_argc, sizeof(char *)); + if (! fuse_argv) { + return INDEXTOOL_OPERATIONAL_ERROR; + } + for (i = 0; i < fuse_argc; ++i) { + fuse_argv[i] = strdup(argv[i]); + if (! fuse_argv[i]) { + return INDEXTOOL_OPERATIONAL_ERROR; + } + } + struct fuse_args args = FUSE_ARGS_INIT(fuse_argc, fuse_argv); + + /* Check for LANG variable and set it to en_US.UTF-8 if it is unset. */ + lang = getenv("LANG"); + if (! lang) { + fprintf(stderr, "LTFS9015W Setting the locale to 'en_US.UTF-8'. If this is wrong, please set the LANG environment variable before starting mkltfs.\n"); + ret = setenv("LANG", "en_US.UTF-8", 1); + if (ret) { + fprintf(stderr, "LTFS9016E Cannot set the LANG environment variable\n"); + return INDEXTOOL_OPERATIONAL_ERROR; + } + } + + /* Start up libltfs with the default logging level. */ +#ifndef mingw_PLATFORM + openlog("ltfsindextool", LOG_PID, LOG_USER); +#endif + ret = ltfs_init(LTFS_INFO, true, false); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10000E, ret); + return INDEXTOOL_OPERATIONAL_ERROR; + } + + /* Setup signal handler to terminate cleanly */ + ret = ltfs_set_signal_handlers(); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10013E); + return INDEXTOOL_OPERATIONAL_ERROR; + } + + /* Register messages with libltfs */ + ret = ltfsprintf_load_plugin("bin_ltfsindextool", bin_ltfsindextool_dat, &message_handle); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10012E, ret); + return INDEXTOOL_OPERATIONAL_ERROR; + } + + /* Set up empty options and load the configuration file. */ + memset(&opt, 0, sizeof(struct indextool_opts)); + opt.blocksize = LTFS_DEFAULT_BLOCKSIZE; + opt.partition = PART_BOTH; + opt.start_pos = START_POS; + opt.out_dir = OUTPUT_DIR; + + /* Check for a config file path given on the command line */ + while (true) { + int option_index = 0; + int c = getopt_long(argc, argv, short_options, long_options, &option_index); + if (c == -1) + break; + if (c == 'i') { + config_file = strdup(optarg); + break; + } + } + + /* Load configuration file */ + ret = config_file_load(config_file, &opt.config); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10008E, ret); + return INDEXTOOL_OPERATIONAL_ERROR; + } + + /* Parse all command line arguments */ + + optind = 1; + int num_of_o = 0; + while (true) { + int option_index = 0; + int c = getopt_long(argc, argv, short_options, long_options, &option_index); + if (c == -1) + break; + + switch (c) { + case 'i': + break; + case 'e': + free(opt.backend_path); + opt.backend_path = strdup(optarg); + break; + case 'd': + opt.devname = strdup(optarg); + break; + case 'p': + opt.partition = atoi(optarg); + break; + case 's': + opt.start_pos = strtoull(optarg, NULL, 0); + break; + case '^': + opt.out_dir = strdup(optarg); + break; + case 'b': + opt.blocksize = atoi(optarg); + break; + case '-': + opt.kmi_backend_name = strdup(optarg); + break; + case 'q': + opt.quiet = true; + break; + case 't': + opt.trace = true; + break; + case '!': + opt.syslogtrace = true; + break; + case 'h': + show_usage(argv[0], opt.config, false); + return 0; + case 'V': + ltfsresult(19546I, "ltfsindextool", PACKAGE_VERSION); + ltfsresult(19546I, "LTFS Format Specification", LTFS_INDEX_VERSION_STR); + return 0; + case '?': + default: + show_usage(argv[0], opt.config, false); + return INDEXTOOL_USAGE_SYNTAX_ERROR; + } + } + + if(argv[optind + num_of_o]) + opt.filename = strdup(argv[optind + num_of_o]); + + if(_indextool_validate_options(argv[0], &opt)) { + return PROG_USAGE_SYNTAX_ERROR; + } + + /* Pick up default backend if one wasn't specified before */ + if (! opt.backend_path) { + const char *default_backend = config_file_get_default_plugin("tape", opt.config); + if (! default_backend) { + ltfsmsg(LTFS_ERR, 10009E); + return INDEXTOOL_OPERATIONAL_ERROR; + } + opt.backend_path = strdup(default_backend); + } + if (! opt.kmi_backend_name) { + const char *default_backend = config_file_get_default_plugin("kmi", opt.config); + if (default_backend) + opt.kmi_backend_name = strdup(default_backend); + else + opt.kmi_backend_name = strdup("none"); + } + if (opt.kmi_backend_name && strcmp(opt.kmi_backend_name, "none") == 0) + opt.kmi_backend_name = NULL; + + /* Set the logging level */ + if (opt.quiet && opt.trace) { + ltfsmsg(LTFS_ERR, 9012E); + show_usage(argv[0], opt.config, false); + return 1; + } else if (opt.quiet) { + log_level = LTFS_WARN; + syslog_level = LTFS_NONE; + } else if (opt.trace) { + log_level = LTFS_DEBUG; + syslog_level = LTFS_NONE; + } else if (opt.syslogtrace) + log_level = syslog_level = LTFS_DEBUG; + else { + log_level = LTFS_INFO; + syslog_level = LTFS_NONE; + } + + ltfs_set_log_level(log_level); + ltfs_set_syslog_level(syslog_level); + + /* Starting ltfsindextool */ + ltfsmsg(LTFS_INFO, 19500I, PACKAGE_NAME, PACKAGE_VERSION, log_level); + + /* Show command line arguments */ + for (i = 0, cmd_args_len = 0 ; i < argc; i++) { + cmd_args_len += strlen(argv[i]) + 1; + } + cmd_args = calloc(1, cmd_args_len + 1); + if (!cmd_args) { + /* Memory allocation failed */ + ltfsmsg(LTFS_ERR, 10001E, "ltfsindextool (arguments)"); + return INDEXTOOL_OPERATIONAL_ERROR; + } + strcat(cmd_args, argv[0]); + for (i = 1; i < argc; i++) { + strcat(cmd_args, " "); + strcat(cmd_args, argv[i]); + } + ltfsmsg(LTFS_INFO, 19542I, cmd_args); + free(cmd_args); + + /* Show build time information */ + ltfsmsg(LTFS_INFO, 19502I, BUILD_SYS_FOR); + ltfsmsg(LTFS_INFO, 19503I, BUILD_SYS_GCC); + + /* Show run time information */ + show_runtime_system_info(); + + /* Actually mkltfs logic starts here */ + ret = ltfs_volume_alloc("dummy", &vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 19501E); + return INDEXTOOL_OPERATIONAL_ERROR; + } + + switch (opt.mode) { + case OP_CHECK: + ret = check_index(vol, &opt, &args); + break; + case OP_CAPTURE: + ret = capture_index(vol, &opt, &args); + break; + default: + ltfsmsg(LTFS_ERR, 19541E); + ret = PROG_USAGE_SYNTAX_ERROR; + break; + } + + /* Cleaning up */ + free(opt.backend_path); + free(opt.kmi_backend_name); + free(opt.devname); + config_file_free(opt.config); + ltfsprintf_unload_plugin(message_handle); + ltfs_finish(); + + return ret; +} From 27e5ea1bf6515689b7745b0d7f71fc730ac9900f Mon Sep 17 00:00:00 2001 From: Scott Talbert Date: Sun, 22 Aug 2021 21:22:10 -0400 Subject: [PATCH 046/121] Fix fallback to pkg-config (#310) This fixes needs to build the LTFS on Debian 11 --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index e317546d..13aa3c41 100644 --- a/configure.ac +++ b/configure.ac @@ -341,7 +341,7 @@ ICU_MODULE_CFLAGS="`icu-config --cppflags 2> /dev/null`"; ICU_MODULE_LIBS="`icu-config --ldflags 2> /dev/null`"; if test -z "$ICU_MODULE_LIBS" then - PKG_CHECK_MODULES([ICU_MODULE], [icu >= 0.21]) + PKG_CHECK_MODULES([ICU_MODULE], [icu-uc >= 0.21]) fi AC_MSG_CHECKING([use latest ICU]) From 33dbad1726cf83003133d0a5c8e3c01c8792942e Mon Sep 17 00:00:00 2001 From: Reid Linnemann <3542659+linnemannr@users.noreply.github.com> Date: Sun, 5 Sep 2021 21:52:45 -0600 Subject: [PATCH 047/121] Correct ltfs_check_medium handling of partitions ending in data+fm (#302) --- src/libltfs/ltfs_internal.c | 89 +++++++++++++++++++++++++++++++++++-- 1 file changed, 86 insertions(+), 3 deletions(-) diff --git a/src/libltfs/ltfs_internal.c b/src/libltfs/ltfs_internal.c index f3d0055d..efdcbdf5 100644 --- a/src/libltfs/ltfs_internal.c +++ b/src/libltfs/ltfs_internal.c @@ -982,6 +982,68 @@ int _ltfs_make_lost_found(tape_block_t ip_eod, tape_block_t dp_eod, return 0; } +/** + * Find the appropriate append address for a partition at which calling ltfs_write_index() will + * correctly restore consistency to the partition. + * + * @param vol The ltfs volume to work on + * @param partid The partition to examine + * @param block The starting block of the final index on the partition + * @param fix allow simple fixes to make the volume consistent? + * @return 0 if eod is the appropriate append address, >0 for the absolute block address to append + * at, <0 on error + */ +static int _ltfs_find_append_blk_after_idx(struct ltfs_volume *vol, char partid, tape_block_t block, bool fix) { + unsigned int n_fm_after = 0; + struct tc_position idx_pos; + struct tc_position final_fm_pos; + int ret = 0; + + idx_pos.partition = ltfs_part_id2num(partid, vol); + idx_pos.block = block; + ret = tape_seek(vol->device, &idx_pos); + if (ret != 0) { + if (partid == vol->label->partid_ip) + ltfsmsg(LTFS_ERR, 11023E); + else { + ltfsmsg(LTFS_ERR, 11020E); + } + goto out; + } + while(ret == 0) { + ret = tape_spacefm(vol->device, 1); + if (ret == 0) { + tape_update_position(vol->device, &final_fm_pos); + n_fm_after++; + } else { + if (ret != -EDEV_EOD_DETECTED) { + goto out; + } + } + } + ret = 0; + switch (n_fm_after){ + case 0: + case 1: + /* Append block does not need to be altered */ + break; + case 2: + /* (index | data | ... eod) - unexpected fm after data, possible incomplete index */ + if (fix) { + ret = final_fm_pos.block-1; + } else { + ret = -LTFS_OP_TO_INV; + } + break; + default: + /* (index | data | ??? | ... eod) - invalid format*/ + ret = -LTFS_OP_TO_INV; + break; + } +out: + return ret; +} + /** * Check a volume for physical consistency. This should be called when there is some doubt about * the validity of the MAM parameters; it reads index files from both partitions and verifies @@ -993,7 +1055,8 @@ int _ltfs_make_lost_found(tape_block_t ip_eod, tape_block_t dp_eod, * * @param fix allow simple fixes to make the tape consistent? * Here, simple means writing an additional logically unmodified copy - * or copies of an index file already present on the tape. + * or copies of an index file already present on the tape. Also + * allows truncating incomplete index at the end of the partition. * @param deep Allow fancy recovery procedures? In particular, this flag enables recovery in the * case where extra blocks (after the last index file on a partition) are found on the * tape. The nature of this recovery is controlled by the recover_extra flag. @@ -1121,8 +1184,28 @@ int ltfs_check_medium(bool fix, bool deep, bool recover_extra, bool recover_syml if (ret < 0) goto out_unlock; - /* Set append position for index partition. */ - if (ip_have_index && ! ip_blocks_after) { + /* Set append position for data partition to end of trailing data. */ + if (dp_have_index && dp_blocks_after) { + ret = _ltfs_find_append_blk_after_idx(vol, vol->label->partid_dp, dp_index->selfptr.block, fix); + if (ret < 0) { + goto out_unlock; + } else { + dp_eod = ret; + check_err(tape_set_append_position(vol->device, dp_num, dp_eod), + 11222E, out_unlock); + } + } + + /* Set append position for index partition to end of trailing data or preceding data */ + if (ip_have_index && ip_blocks_after) { + ret = _ltfs_find_append_blk_after_idx(vol, vol->label->partid_ip, ip_index->selfptr.block, fix); + if (ret <0) { + goto out_unlock; + } else { + ip_eod = ret; + check_err(tape_set_append_position(vol->device, ip_num, ip_eod), + 11222E, out_unlock); + } check_err(tape_set_append_position(vol->device, ip_num, ip_index->selfptr.block - 1), 11222E, out_unlock); } From 618859c05444768e13963812918f22f5b062bd73 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 10 Sep 2021 20:32:40 +0900 Subject: [PATCH 048/121] Modify build configuration (#311) This commit includes the changes below. - Check ICU version and use unorm2 function automatically - Move to new lock architecture --- configure.ac | 40 +++++++++++++++++++--------------------- src/libltfs/pathname.c | 12 ++++++------ 2 files changed, 25 insertions(+), 27 deletions(-) diff --git a/configure.ac b/configure.ac index 13aa3c41..649da0c6 100644 --- a/configure.ac +++ b/configure.ac @@ -152,22 +152,13 @@ AC_ARG_ENABLE([snmp], AC_MSG_RESULT([$snmp]) dnl -dnl Handle --enable-new-locking (default: depend on platform) -dnl NetBSD: old locking is broken, force new locking -dnl others: default to old locking +dnl Handle --enable-new-locking (default: yes) dnl -if test "x${host_netbsd}" = "xyes" -then - use_new_locking_default=yes -else - use_new_locking_default=no -fi - AC_MSG_CHECKING([whether to enable new locking system or not]) AC_ARG_ENABLE([new-locking], [AS_HELP_STRING([--enable-new-locking],[Use new locking system or not])], [use_new_locking=$enableval], - [use_new_locking=$use_new_locking_default] + [use_new_locking=yes] ) AC_MSG_RESULT([$use_new_locking]) @@ -337,16 +328,20 @@ fi dnl dnl Check for ICU dnl -ICU_MODULE_CFLAGS="`icu-config --cppflags 2> /dev/null`"; +AC_MSG_CHECKING(ICU version) +ICU_MODULE_VERSION="`icu-config --version 2> /dev/null`"; +ICU_MODULE_CFLAGS=" `icu-config --cppflags 2> /dev/null`"; ICU_MODULE_LIBS="`icu-config --ldflags 2> /dev/null`"; -if test -z "$ICU_MODULE_LIBS" +AC_MSG_RESULT([$ICU_MODULE_VERSION]) + +if test -z "$ICU_MODULE_VERSION" then PKG_CHECK_MODULES([ICU_MODULE], [icu-uc >= 0.21]) fi -AC_MSG_CHECKING([use latest ICU]) +AC_MSG_CHECKING([for using ICU6x (unorm2) functions forcibly]) AC_ARG_ENABLE([icu_6x], - [AS_HELP_STRING([--enable-icu-6x],[Support handling of ICU 6x functions])], + [AS_HELP_STRING([--enable-icu-6x],[Force to use ICU6x (unorm2) functions])], [icu_6x=$enableval], [icu_6x=no] ) @@ -354,13 +349,16 @@ AC_MSG_RESULT([$icu_6x]) if test "x${icu_6x}" = "xyes" then - AC_MSG_CHECKING(for ICU version) - ICU_MODULE_VERSION="`icu-config --version 2> /dev/null`"; - if test "${ICU_MODULE_VERSION%%.*}" -ge "60" + AM_EXTRA_CPPFLAGS="${AM_EXTRA_CPPFLAGS} -D USE_UNORM2" +else + AC_MSG_CHECKING(for using unorm2 functions) + if test "${ICU_MODULE_VERSION%%.*}" -ge "56" then - AM_EXTRA_CPPFLAGS="${AM_EXTRA_CPPFLAGS} -D ICU6x" + AM_EXTRA_CPPFLAGS="${AM_EXTRA_CPPFLAGS} -D USE_UNORM2" + AC_MSG_RESULT(yes) + else + AC_MSG_RESULT(no) fi - AC_MSG_RESULT([$ICU_MODULE_VERSION]) fi dnl @@ -483,7 +481,7 @@ fi if test "x$warning_as_error" = "xyes" then - AM_CFLAGS="${AM_CFLAGS} -Werror -Wno-deprecated-declarations" + AM_CFLAGS="${AM_CFLAGS} -Werror" fi if test "x$use_new_locking" = "xyes" diff --git a/src/libltfs/pathname.c b/src/libltfs/pathname.c index ca29c7b7..de76c0aa 100644 --- a/src/libltfs/pathname.c +++ b/src/libltfs/pathname.c @@ -57,7 +57,7 @@ #include #include #include -#ifdef ICU6x +#ifdef USE_UNORM2 #include #else #include @@ -67,7 +67,7 @@ #include #include #include -#ifdef ICU6x +#ifdef USE_UNORM2 #include #else #include @@ -649,13 +649,13 @@ int _pathname_foldcase_icu(const UChar *src, UChar **dest) } /** - * ICU5x/ICU6x handle: gets a reference to a singleton unorm2 object (ICU6x) or to a constant + * ICU5x/USE_UNORM2 handle: gets a reference to a singleton unorm2 object (USE_UNORM2) or to a constant * value that we can use to differentiate between NFC and NFD modes (ICU5x). Neither should be * freed after use. */ static inline void *_unorm_handle(bool nfc, UErrorCode *err) { -#ifdef ICU6x +#ifdef USE_UNORM2 *err = U_ZERO_ERROR; return (void *) unorm2_getInstance(NULL, "nfc", nfc ? UNORM2_COMPOSE : UNORM2_DECOMPOSE, err); #else @@ -666,7 +666,7 @@ static inline void *_unorm_handle(bool nfc, UErrorCode *err) static inline UNormalizationCheckResult _unorm_quickCheck(void *handle, const UChar *src, UChar **dest, UErrorCode *err) { *err = U_ZERO_ERROR; -#ifdef ICU6x +#ifdef USE_UNORM2 const UNormalizer2 *n2 = (const UNormalizer2 *) handle; return unorm2_quickCheck(n2, src, -1, err); #else @@ -678,7 +678,7 @@ static inline UNormalizationCheckResult _unorm_quickCheck(void *handle, const UC static inline int32_t _unorm_normalize(void *handle, const UChar *src, UChar **dest, int32_t len, UErrorCode *err) { *err = U_ZERO_ERROR; -#ifdef ICU6x +#ifdef USE_UNORM2 const UNormalizer2 *n2 = (const UNormalizer2 *) handle; return unorm2_normalize(n2, src, -1, dest ? *dest : NULL, len, err); #else From 22be24e88a252f5a325a6bacd875c28f05162029 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 28 Sep 2021 14:15:39 +0900 Subject: [PATCH 049/121] Enable code scan by CodeQL (Experimental) --- .github/workflows/codeql-analysis.yml | 71 +++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 .github/workflows/codeql-analysis.yml diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml new file mode 100644 index 00000000..f8aedb58 --- /dev/null +++ b/.github/workflows/codeql-analysis.yml @@ -0,0 +1,71 @@ +# For most projects, this workflow file will not need changing; you simply need +# to commit it to your repository. +# +# You may wish to alter this file to override the set of languages analyzed, +# or to provide custom queries or build logic. +# +# ******** NOTE ******** +# We have attempted to detect the languages in your repository. Please check +# the `language` matrix defined below to confirm you have the correct set of +# supported CodeQL languages. +# +name: "CodeQL" + +on: + push: + branches: [ master, v2* ] + pull_request: + # The branches below must be a subset of the branches above + branches: [ master ] + schedule: + - cron: '26 12 * * 0' + +jobs: + analyze: + name: Analyze + runs-on: ubuntu-latest + permissions: + actions: read + contents: read + security-events: write + + strategy: + fail-fast: false + matrix: + language: [ 'cpp', 'python' ] + # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ] + # Learn more: + # https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed + + steps: + - name: Checkout repository + uses: actions/checkout@v2 + + # Initializes the CodeQL tools for scanning. + - name: Initialize CodeQL + uses: github/codeql-action/init@v1 + with: + languages: ${{ matrix.language }} + # If you wish to specify custom queries, you can do so here or in a config file. + # By default, queries listed here will override any specified in a config file. + # Prefix the list here with "+" to use these queries and those in the config file. + # queries: ./path/to/local/query, your-org/your-repo/queries@main + + # Autobuild attempts to build any compiled languages (C/C++, C#, or Java). + # If this step fails, then you should remove it and run the build manually (see below) + - name: Autobuild + uses: github/codeql-action/autobuild@v1 + + # ℹ️ Command-line programs to run using the OS shell. + # 📚 https://git.io/JvXDl + + # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines + # and modify them (or add more) to build your code if your project + # uses a compiled language + + #- run: | + # make bootstrap + # make release + + - name: Perform CodeQL Analysis + uses: github/codeql-action/analyze@v1 From 226e76b5f0abd17ccd97f34deb8f1a1cd70d0749 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 28 Sep 2021 14:24:33 +0900 Subject: [PATCH 050/121] Update CodeQL workflow Do not use predefined `Autobuild` configuration because it doesn't work. --- .github/workflows/codeql-analysis.yml | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index f8aedb58..4253f79c 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -53,8 +53,8 @@ jobs: # Autobuild attempts to build any compiled languages (C/C++, C#, or Java). # If this step fails, then you should remove it and run the build manually (see below) - - name: Autobuild - uses: github/codeql-action/autobuild@v1 + #- name: Autobuild + # uses: github/codeql-action/autobuild@v1 # ℹ️ Command-line programs to run using the OS shell. # 📚 https://git.io/JvXDl @@ -63,9 +63,10 @@ jobs: # and modify them (or add more) to build your code if your project # uses a compiled language - #- run: | - # make bootstrap - # make release + - run: | + ./autogen.sh + ./configure + make - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 From 60ff79fc00d7b83f245ee0699199d43dcd511884 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 28 Sep 2021 14:30:40 +0900 Subject: [PATCH 051/121] Install prereqs at CodeQL scan --- .github/workflows/codeql-analysis.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 4253f79c..319d0e8b 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -64,6 +64,8 @@ jobs: # uses a compiled language - run: | + apt -q -y update && apt -q -y upgrade && apt -q -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev + ls .github/workflows ./autogen.sh ./configure make From a1724569a19707017a0480a8583e908cae744f88 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 28 Sep 2021 14:35:19 +0900 Subject: [PATCH 052/121] Use apt-get in the CodeQL job --- .github/workflows/codeql-analysis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 319d0e8b..9efcc38b 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -64,7 +64,7 @@ jobs: # uses a compiled language - run: | - apt -q -y update && apt -q -y upgrade && apt -q -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev + apt-get update -q -y update && apt -q -y upgrade && apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev ls .github/workflows ./autogen.sh ./configure From b446455cc4771bd4732aa1005d1330a0f82546cb Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 28 Sep 2021 14:37:49 +0900 Subject: [PATCH 053/121] Fix command error on the CodeQL job --- .github/workflows/codeql-analysis.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 9efcc38b..fb513a53 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -64,7 +64,9 @@ jobs: # uses a compiled language - run: | - apt-get update -q -y update && apt -q -y upgrade && apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev + apt-get -q -y update + apt-get -q -y upgrade + apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev ls .github/workflows ./autogen.sh ./configure From c5a098208dd93d568403a749d862f6d0f6f23d3b Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 28 Sep 2021 14:40:00 +0900 Subject: [PATCH 054/121] Run apt-get on root in the CodeQL job --- .github/workflows/codeql-analysis.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index fb513a53..7cc8520b 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -64,9 +64,9 @@ jobs: # uses a compiled language - run: | - apt-get -q -y update - apt-get -q -y upgrade - apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev + sudo apt-get -q -y update + sudo apt-get -q -y upgrade + sudo apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev ls .github/workflows ./autogen.sh ./configure From 4818e0acab6a362758ea140a407b9860aae944ee Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 28 Sep 2021 14:48:25 +0900 Subject: [PATCH 055/121] Add dummy icu-config for the CodeQL job --- .github/workflows/codeql-analysis.yml | 4 ++-- .github/workflows/icu-config | 12 ++++++++++++ 2 files changed, 14 insertions(+), 2 deletions(-) create mode 100755 .github/workflows/icu-config diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 7cc8520b..511554d1 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -67,10 +67,10 @@ jobs: sudo apt-get -q -y update sudo apt-get -q -y upgrade sudo apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev - ls .github/workflows + sudo cp .github/workflows/icu-config /usr/bin/icu-config ./autogen.sh ./configure - make + make - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 diff --git a/.github/workflows/icu-config b/.github/workflows/icu-config new file mode 100755 index 00000000..d5aed413 --- /dev/null +++ b/.github/workflows/icu-config @@ -0,0 +1,12 @@ +#!/bin/sh + +opts=$1 + +case $opts in + '--cppflags') + echo '' ;; + '--ldflags') + echo '-licuuc -licudata' ;; + *) + echo '/usr/lib/x86_64-linux-gnu/icu/pkgdata.inc' ;; +esac From b5fd43a565b582e882ca58882c9faa87d6e637b6 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 1 Oct 2021 14:47:19 +0900 Subject: [PATCH 056/121] Test for the CodeQL job --- .github/workflows/codeql-analysis.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 511554d1..484bb69d 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -70,7 +70,6 @@ jobs: sudo cp .github/workflows/icu-config /usr/bin/icu-config ./autogen.sh ./configure - make - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 From 2d7fc01ea860887c4823150ccbc0df83430740b7 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 1 Oct 2021 14:51:42 +0900 Subject: [PATCH 057/121] Test for the CodeQL job #2 --- .github/workflows/codeql-analysis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 484bb69d..ddd83ea6 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -70,6 +70,7 @@ jobs: sudo cp .github/workflows/icu-config /usr/bin/icu-config ./autogen.sh ./configure + codeql database create cpp-database --language=cpp --command=make - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 From e845fef9537185a3a9610a6dd40fdfc7952a53c3 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 1 Oct 2021 14:54:29 +0900 Subject: [PATCH 058/121] Anyway, just need to make the tree for CodeQL --- .github/workflows/codeql-analysis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index ddd83ea6..511554d1 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -70,7 +70,7 @@ jobs: sudo cp .github/workflows/icu-config /usr/bin/icu-config ./autogen.sh ./configure - codeql database create cpp-database --language=cpp --command=make + make - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 From bc5ce0e358271856c95363f3205899b19fbf4451 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 1 Oct 2021 16:03:23 +0900 Subject: [PATCH 059/121] Add autobuild script for CodeQL analysis --- .github/workflows/codeql-analysis.yml | 22 ++++++++++++---------- build.sh | 5 +++++ 2 files changed, 17 insertions(+), 10 deletions(-) create mode 100755 build.sh diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 511554d1..09893b8b 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -53,8 +53,8 @@ jobs: # Autobuild attempts to build any compiled languages (C/C++, C#, or Java). # If this step fails, then you should remove it and run the build manually (see below) - #- name: Autobuild - # uses: github/codeql-action/autobuild@v1 + - name: Autobuild + uses: github/codeql-action/autobuild@v1 # ℹ️ Command-line programs to run using the OS shell. # 📚 https://git.io/JvXDl @@ -63,14 +63,16 @@ jobs: # and modify them (or add more) to build your code if your project # uses a compiled language - - run: | - sudo apt-get -q -y update - sudo apt-get -q -y upgrade - sudo apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev - sudo cp .github/workflows/icu-config /usr/bin/icu-config - ./autogen.sh - ./configure - make + #- run: | + # sudo apt-get -q -y update + # sudo apt-get -q -y upgrade + # sudo apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev + # sudo cp .github/workflows/icu-config /usr/bin/icu-config + # ./autogen.sh + # ./configure + # make - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 + with: + queries: /home/runner/work/_temp/codeql_databases/cpp-queries-builtin.qls diff --git a/build.sh b/build.sh new file mode 100755 index 00000000..97fcf6c4 --- /dev/null +++ b/build.sh @@ -0,0 +1,5 @@ +#!/bin/sh + +./autogen.sh +./configure +make From 39fdc49480a36aacaa9680a55eaff8c6e3d88928 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 1 Oct 2021 16:11:27 +0900 Subject: [PATCH 060/121] Fix autobuild script for CodeQL analysis --- .github/workflows/codeql-analysis.yml | 7 ++++++- build.sh | 7 +++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 09893b8b..acd67fdc 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -72,7 +72,12 @@ jobs: # ./configure # make - - name: Perform CodeQL Analysis + - if: matrix.language == 'cpp' + name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 with: queries: /home/runner/work/_temp/codeql_databases/cpp-queries-builtin.qls + + - if: matrix.language == 'python' + name: Perform CodeQL Analysis + uses: github/codeql-action/analyze@v1 diff --git a/build.sh b/build.sh index 97fcf6c4..8875035a 100755 --- a/build.sh +++ b/build.sh @@ -1,5 +1,12 @@ #!/bin/sh +# This is the script for building the tree under CodeQL scanning in github flow. +# Do not use on other environments + +sudo apt-get -q -y update +sudo apt-get -q -y upgrade +sudo apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev +sudo cp .github/workflows/icu-config /usr/bin/icu-config ./autogen.sh ./configure make From 60b202442b580ec45c8e5cb6c110e4e7a6f9d2d9 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 1 Oct 2021 16:25:13 +0900 Subject: [PATCH 061/121] Use default built-in queries --- .github/workflows/codeql-analysis.yml | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index acd67fdc..64e17ba6 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -72,12 +72,5 @@ jobs: # ./configure # make - - if: matrix.language == 'cpp' - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@v1 - with: - queries: /home/runner/work/_temp/codeql_databases/cpp-queries-builtin.qls - - - if: matrix.language == 'python' - name: Perform CodeQL Analysis + -name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 From 9de617aaaa0a4cd15e2c3fffcc78d69dbd2b04f1 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 1 Oct 2021 16:29:41 +0900 Subject: [PATCH 062/121] Use default built-in queries --- .github/workflows/codeql-analysis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 64e17ba6..4dec8b88 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -72,5 +72,5 @@ jobs: # ./configure # make - -name: Perform CodeQL Analysis + - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v1 From 55656a7783dda9ef35217c242c7809e83a3bc7fb Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 18 Oct 2021 11:19:38 +0900 Subject: [PATCH 063/121] Return error when index write on DP is failed at sync (#316) --- messages/internal_error/root.txt | 1 + src/libltfs/arch/errormap.c | 3 ++- src/libltfs/ltfs.c | 16 +++++++++++++++- src/libltfs/ltfs_error.h | 1 + 4 files changed, 19 insertions(+), 2 deletions(-) diff --git a/messages/internal_error/root.txt b/messages/internal_error/root.txt index cd9fafac..0df5c020 100644 --- a/messages/internal_error/root.txt +++ b/messages/internal_error/root.txt @@ -255,6 +255,7 @@ root:table { I1202E:string{ "Failed to open a stats DB." } I1203E:string{ "There is no trailing FM after an index." } I1204E:string{ "Failed to update safename." } + I1205E:string{ "Unwritten file contents exists in sync." } // Error codes for the XML parser I5000E:string{ "Error happens in reading XML node." } diff --git a/src/libltfs/arch/errormap.c b/src/libltfs/arch/errormap.c index 4d049c6d..9c5f34b6 100644 --- a/src/libltfs/arch/errormap.c +++ b/src/libltfs/arch/errormap.c @@ -288,8 +288,9 @@ static struct error_map fuse_error_list[] = { { LTFS_NEED_START_OVER, "I1200E", EINVAL}, { LTFS_LOCATE_ERROR, "I1201E", EIO}, { LTFS_STATS_DB_OPEN, "I1202E", EIO}, - { LTFS_SAFENAME_FAIL, "I1204E", EINVAL}, { LTFS_NO_TRAIL_FM, "I1203E", EINVAL}, + { LTFS_SAFENAME_FAIL, "I1204E", EINVAL}, + { LTFS_SYNC_FAIL_ON_DP, "I1205E", EIO}, { LTFS_XML_READ_FAIL, "I5000E", EINVAL}, { LTFS_XML_CONST_FAIL, "I5001E", EINVAL}, { LTFS_XML_WRONG_NODE, "I5002E", EINVAL}, diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 2f05a5f5..f562d814 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -3532,10 +3532,24 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) } ret = ltfs_write_index(partition, reason, vol); if (IS_WRITE_PERM(-ret) && partition == ltfs_dp_id(vol)) { + /* + * TODO: Need to determine the last record on DP of the tape and cleanup + * all extents on the volume. Because this write perm have a chance + * to happen in the cases below. + * + * 1. File contents lastly written (and got a write perm on tape drive's flush) + * 2. Index written in ltfs_write_index() + * + * Extents cleaning is required in case1, otherwise LTFS writes an index + * which has unwritten record on the tape. + * + * For now, write the same index on IP and return an error against a sync + * request. + */ ret_r = ltfs_write_index(ltfs_ip_id(vol), SYNC_WRITE_PERM, vol); if (!ret_r) { ltfsmsg(LTFS_INFO, 11344I, bc_print); - ret = ret_r; + ret = LTFS_SYNC_FAIL_ON_DP; } else { ltfsmsg(LTFS_ERR, 11345E, bc_print); ltfsmsg(LTFS_ERR, 11346E, bc_print); diff --git a/src/libltfs/ltfs_error.h b/src/libltfs/ltfs_error.h index acf7b20d..ab661262 100644 --- a/src/libltfs/ltfs_error.h +++ b/src/libltfs/ltfs_error.h @@ -261,6 +261,7 @@ #define LTFS_STATS_DB_OPEN 1202 /* Failed to open a stats DB */ #define LTFS_NO_TRAIL_FM 1203 /* There is no trailing FM after an index */ #define LTFS_SAFENAME_FAIL 1204 /* Failed to update safename */ +#define LTFS_SYNC_FAIL_ON_DP 1205 /* Unwritten file contents exists in sync */ /* * Error codes for the XML parser From 6cdbad017f5921ab7901a135ec65001f6e1b8051 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 18 Oct 2021 13:41:19 +0900 Subject: [PATCH 064/121] Fix potential buffer overrun (#317) --- src/libltfs/xml_reader_libltfs.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/libltfs/xml_reader_libltfs.c b/src/libltfs/xml_reader_libltfs.c index bec3d56a..4935dcd4 100644 --- a/src/libltfs/xml_reader_libltfs.c +++ b/src/libltfs/xml_reader_libltfs.c @@ -95,8 +95,9 @@ static int decode_entry_name(char **new_name, const char *name) CHECK_ARG_NULL(name, -LTFS_NULL_ARG); + /* Always, length must be shorter than original but allocate null termination space */ len = strlen(name); - tmp_name = malloc(len * sizeof(UChar)); + tmp_name = malloc((len * sizeof(UChar)) + 1); buf_decode[2] = '\0'; while (i < len) { From 4c3a760e2d3ea75944df688f74e1ed767ca2cf7a Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 18 Oct 2021 14:28:08 +0900 Subject: [PATCH 065/121] Fix a toctou race condition (#318) --- src/libltfs/arch/arch_info.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/libltfs/arch/arch_info.c b/src/libltfs/arch/arch_info.c index 7a1fe1dc..e0577593 100644 --- a/src/libltfs/arch/arch_info.c +++ b/src/libltfs/arch/arch_info.c @@ -112,18 +112,19 @@ void show_runtime_system_info(void) } strcat(path, "/etc/"); strcat(path, dent->d_name); - if(stat(path, &stat_rel) != -1 && S_ISREG(stat_rel.st_mode)) { - fd = open(path, O_RDONLY); - if( fd == -1) { - ltfsmsg(LTFS_WARN, 17088W); - } else { + fd = open(path, O_RDONLY); + if( fd == -1) { + ltfsmsg(LTFS_WARN, 17088W); + } else { + if (fstat(fd, &stat_rel) != -1 && S_ISREG(stat_rel.st_mode)) { memset(destribution, 0, sizeof(destribution)); read(fd, destribution, sizeof(destribution)); if((tmp = strchr(destribution, '\n')) != NULL) *tmp = '\0'; ltfsmsg(LTFS_INFO, 17089I, destribution); - close(fd); } + + close(fd); } free(path); } From b193b4564c87efc8525b22e44d36961d82eec914 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 8 Nov 2021 16:00:15 +0900 Subject: [PATCH 066/121] Update supporting drives. --- README.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 7bce62d1..4bfbff5a 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,5 @@ ![](https://img.shields.io/github/issues/lineartapefilesystem/ltfs.svg) ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS7%20Build%20Job/badge.svg?branch=master) -[![CodeFactor](https://www.codefactor.io/repository/github/lineartapefilesystem/ltfs/badge)](https://www.codefactor.io/repository/github/lineartapefilesystem/ltfs) [![BSD License](http://img.shields.io/badge/license-BSD-blue.svg?style=flat)](LICENSE) # About this branch @@ -23,6 +22,7 @@ At this time, the target of this project to meet is the [LTFS format specificati | IBM | LTO6 | None | | IBM | LTO7 | None | | IBM | LTO8 | HB81 | + | IBM | LTO9 | None | | IBM | TS1140 | 3694 | | IBM | TS1150 | None | | IBM | TS1155 | None | @@ -31,10 +31,12 @@ At this time, the target of this project to meet is the [LTFS format specificati | HP | LTO6 | T.B.D. | | HP | LTO7 | T.B.D. | | HP | LTO8 | T.B.D. | + | HP | LTO9 | T.B.D. | | Quantum | LTO5 (Only Half Height) | T.B.D. | | Quantum | LTO6 (Only Half Height) | T.B.D. | | Quantum | LTO7 (Only Half Height) | T.B.D. | | Quantum | LTO8 (Only Half Height) | T.B.D. | + | Quantum | LTO9 (Only Half Height) | T.B.D. | ## LTFS Format Specifications @@ -147,9 +149,9 @@ In some systems, you might need `sudo ldconfig -v` after `make install` to load | CentOS 7 | ppc64le | OK - Not checked automatically | | Fedora 28 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Fedora28%20Build%20Job/badge.svg?branch=master) | | Ubuntu 16.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2016.04%20Build%20Job/badge.svg?branch=master) | - | Ubuntu 16.04 LTS | ppc64le | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | + | Ubuntu 16.04 LTS | ppc64le | OK - Not checked automatically | | Ubuntu 18.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2018.04%20Build%20Job/badge.svg?branch=master) | - | Ubuntu 18.04 LTS | ppc64le | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | + | Ubuntu 18.04 LTS | ppc64le | OK - Not checked automatically | | Ubuntu 20.04 LTS (Need icu-config) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2020.04%20Build%20Job/badge.svg?branch=master) | | Debian 9 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian9%20Build%20Job/badge.svg?branch=master) | | Debian 10 (Need icu-config) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian10%20Build%20Job/badge.svg?branch=master) | @@ -197,7 +199,7 @@ make install | OS | Xcode | Package system | Status | |:-------------:|:-----:|:--------------:|:-------------------------------------------------------------------------------------------------------------------------------------:| - | macOS 10.14.6 | 11.3 | Homebrew | [![Build Status](https://travis-ci.org/LinearTapeFileSystem/ltfs.svg?branch=master)](https://travis-ci.org/LinearTapeFileSystem/ltfs) | + | macOS 10.14.6 | 11.3 | Homebrew | OK - Not checked automatically | | macOS 10.15 | 12.4 | Homebrew | OK - Not checked automatically | | macOS 11 | 12.4 | Homebrew | OK - Not checked automatically | @@ -217,7 +219,7 @@ make install | Version | Arch | Status | |:-------:|:-------:|:------------------------------:| | 11 | x86\_64 | OK - Not checked automatically | - | 12 | x86\_64 | OK - Not checked automatically | + | 12 | x86\_64 | OK - Not checked automatically | ### Build and install on NetBSD From 77e59f294fd91507d67e7833240518c191264d3f Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 19 Nov 2021 15:05:44 +0900 Subject: [PATCH 067/121] Tiny correction for sync fail (#319) --- src/libltfs/ltfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index f562d814..dc215bc0 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -3549,7 +3549,7 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) ret_r = ltfs_write_index(ltfs_ip_id(vol), SYNC_WRITE_PERM, vol); if (!ret_r) { ltfsmsg(LTFS_INFO, 11344I, bc_print); - ret = LTFS_SYNC_FAIL_ON_DP; + ret = -LTFS_SYNC_FAIL_ON_DP; } else { ltfsmsg(LTFS_ERR, 11345E, bc_print); ltfsmsg(LTFS_ERR, 11346E, bc_print); From 0a821232360d51761988ef78d8355b28308cd813 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 22 Nov 2021 15:59:14 +0900 Subject: [PATCH 068/121] Support `ltfs.hash.*` EAs (#320) This commit also includes the refactoring of `xattr.c` --- src/libltfs/ltfs_fsops.c | 12 +- src/libltfs/xattr.c | 1324 +++++++++++++++++++------------------- src/libltfs/xattr.h | 11 +- 3 files changed, 677 insertions(+), 670 deletions(-) diff --git a/src/libltfs/ltfs_fsops.c b/src/libltfs/ltfs_fsops.c index e426dbfb..89fa1e50 100644 --- a/src/libltfs/ltfs_fsops.c +++ b/src/libltfs/ltfs_fsops.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2021 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -48,6 +48,10 @@ ** IBM Almaden Research Center ** lucasvr@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -1037,7 +1041,7 @@ int ltfs_fsops_setxattr(const char *path, const char *name, const char *value, s goto out_free; } - new_name_strip = _xattr_strip_name(new_name); + new_name_strip = xattr_strip_name(new_name); if (! new_name_strip) { /* Namespace is not supported (Linux) */ ret = -LTFS_XATTR_NAMESPACE; @@ -1153,7 +1157,7 @@ int ltfs_fsops_getxattr(const char *path, const char *name, char *value, size_t ltfsmsg(LTFS_ERR, 11125E, ret); goto out_free; } - new_name_strip = _xattr_strip_name(new_name); + new_name_strip = xattr_strip_name(new_name); if (! new_name_strip) { /* Namespace is not supported (Linux) */ ret = -LTFS_NO_XATTR; @@ -1305,7 +1309,7 @@ int ltfs_fsops_removexattr(const char *path, const char *name, ltfs_file_id *id, ltfsmsg(LTFS_ERR, 11137E, ret); goto out_free; } - new_name_strip = _xattr_strip_name(new_name); + new_name_strip = xattr_strip_name(new_name); if (! new_name_strip) { /* Namespace is not supported (Linux) */ ret = -LTFS_NO_XATTR; diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index 40281028..bcf85b6b 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2021 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -48,6 +48,10 @@ ** IBM Almaden Research Center ** lucasvr@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -64,598 +68,301 @@ #include "ltfs_internal.h" #include "arch/time_internal.h" -int _xattr_seek(struct xattr_info **out, struct dentry *d, const char *name); -int _xattr_lock_dentry(const char *name, bool modify, struct dentry *d, struct ltfs_volume *vol); -void _xattr_unlock_dentry(const char *name, bool modify, struct dentry *d, struct ltfs_volume *vol); -const char *_xattr_strip_name(const char *name); -int _xattr_list_physicals(struct dentry *d, char *list, size_t size); -bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_volume *vol); -int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, const char *name, - struct ltfs_volume *vol); -int _xattr_set_virtual(struct dentry *d, const char *name, const char *value, - size_t size, struct ltfs_volume *vol); -int _xattr_remove_virtual(struct dentry *d, const char *name, struct ltfs_volume *vol); -bool _xattr_is_worm_ea(const char *name); - /* Helper functions for formatting virtual EA output */ -int _xattr_get_cartridge_health(cartridge_health_info *h, int64_t *val, char **outval, - const char *msg, struct ltfs_volume *vol); -int _xattr_get_cartridge_health_u64(cartridge_health_info *h, uint64_t *val, char **outval, - const char *msg, struct ltfs_volume *vol); -int _xattr_get_cartridge_capacity(struct device_capacity *cap, unsigned long *val, char **outval, - const char *msg, struct ltfs_volume *vol); -int _xattr_get_time(struct ltfs_timespec *val, char **outval, const char *msg); -int _xattr_get_dentry_time(struct dentry *d, struct ltfs_timespec *val, char **outval, - const char *msg); -int _xattr_get_string(const char *val, char **outval, const char *msg); -int _xattr_get_tapepos(struct tape_offset *val, char **outval, const char *msg); -int _xattr_get_partmap(struct ltfs_label *label, char **outval, const char *msg); -int _xattr_get_version(int version, char **outval, const char *msg); - -int _xattr_set_time(struct dentry *d, struct ltfs_timespec *out, const char *value, size_t size, - const char *msg, struct ltfs_volume *vol); - -int _xattr_get_vendorunique_xattr(char **outval, const char *msg, struct ltfs_volume *vol); -int _xattr_set_vendorunique_xattr(const char *name, const char *value, size_t size, struct ltfs_volume *vol); - -int xattr_do_set(struct dentry *d, const char *name, const char *value, size_t size, - struct xattr_info *xattr) +static int _xattr_get_cartridge_health(cartridge_health_info *h, int64_t *val, char **outval, + const char *msg, struct ltfs_volume *vol) { - int ret = 0; - - /* clear existing xattr or set up new one */ - if (xattr) { - if (xattr->value) { - free(xattr->value); - xattr->value = NULL; - } - } else { - xattr = (struct xattr_info *) calloc(1, sizeof(struct xattr_info)); - if (! xattr) { - ltfsmsg(LTFS_ERR, 10001E, "xattr_do_set: xattr"); + int ret = ltfs_get_cartridge_health(h, vol); + if (ret == 0) { + ret = asprintf(outval, "%"PRId64, *val); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10001E, msg); + *outval = NULL; return -LTFS_NO_MEMORY; } - xattr->key.name = strdup(name); - if (! xattr->key.name) { - ltfsmsg(LTFS_ERR, 10001E, "xattr_do_set: xattr key"); + } else + *outval = NULL; + return ret; +} + +static int _xattr_get_cartridge_health_u64(cartridge_health_info *h, uint64_t *val, char **outval, + const char *msg, struct ltfs_volume *vol) +{ + int ret = ltfs_get_cartridge_health(h, vol); + if (ret == 0 && (int64_t)(*val) != UNSUPPORTED_CARTRIDGE_HEALTH) { + ret = asprintf(outval, "%"PRIu64, *val); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10001E, msg); + *outval = NULL; ret = -LTFS_NO_MEMORY; - goto out_free; } - xattr->key.percent_encode = fs_is_percent_encode_required(xattr->key.name); - TAILQ_INSERT_HEAD(&d->xattrlist, xattr, list); - } - - /* copy new value */ - xattr->size = size; - if (size > 0) { - xattr->value = (char *)malloc(size); - if (! xattr->value) { - ltfsmsg(LTFS_ERR, 10001E, "xattr_do_set: xattr value"); + } else if (ret == 0) { + ret = asprintf(outval, "%"PRId64, UNSUPPORTED_CARTRIDGE_HEALTH); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10001E, msg); + *outval = NULL; ret = -LTFS_NO_MEMORY; - goto out_remove; } - memcpy(xattr->value, value, size); - } - return 0; + } else + *outval = NULL; + return ret; +} -out_remove: - TAILQ_REMOVE(&d->xattrlist, xattr, list); -out_free: - if (xattr->key.name) - free(xattr->key.name); - free(xattr); +static int _xattr_get_cartridge_capacity(struct device_capacity *cap, unsigned long *val, + char **outval, const char *msg, struct ltfs_volume *vol) +{ + double scale = vol->label->blocksize / 1048576.0; + int ret = ltfs_capacity_data_unlocked(cap, vol); + if (ret == 0) { + ret = asprintf(outval, "%lu", (unsigned long)((*val) * scale)); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10001E, msg); + *outval = NULL; + return -LTFS_NO_MEMORY; + } + } else + *outval = NULL; return ret; } -/** - * Set an extended attribute. - * @param d File or directory to set the xattr on. - * @param name Name to set. - * @param value Value to set, may be binary, not necessarily null-terminated. - * @param size Size of value in bytes. - * @param flags XATTR_REPLACE to fail if xattr doesn't exist, XATTR_CREATE to fail if it does - * exist, or 0 to ignore any existing value. - * @return 0 on success or a negative value on error. - */ -int xattr_set(struct dentry *d, const char *name, const char *value, size_t size, - int flags, struct ltfs_volume *vol) +static int _xattr_get_time(struct ltfs_timespec *val, char **outval, const char *msg) { - struct xattr_info *xattr; - bool replace, create; int ret; - bool is_worm_cart = false; - bool disable_worm_ea = false; - char *new_value="1"; - bool write_idx = false; - - CHECK_ARG_NULL(d, -LTFS_NULL_ARG); - CHECK_ARG_NULL(name, -LTFS_NULL_ARG); - CHECK_ARG_NULL(value, -LTFS_NULL_ARG); - CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); - if (size > LTFS_MAX_XATTR_SIZE) - return -LTFS_LARGE_XATTR; /* this is the error returned by ext3 when the xattr is too large */ - replace = flags & XATTR_REPLACE; - create = flags & XATTR_CREATE; + ret = xml_format_time(*val, outval); + if (! (*outval)) { + ltfsmsg(LTFS_ERR, 11145E, msg); + return -LTFS_NO_MEMORY; + } - ret = _xattr_lock_dentry(name, true, d, vol); - if (ret < 0) - return ret; + return ret; +} - ret = tape_get_worm_status(vol->device, &is_worm_cart); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 17237E, "set xattr: cart stat"); - ret = -LTFS_XATTR_ERR; - goto out_unlock; - } +static int _xattr_get_dentry_time(struct dentry *d, struct ltfs_timespec *val, char **outval, + const char *msg) +{ + int ret; + acquireread_mrsw(&d->meta_lock); + ret = _xattr_get_time(val, outval, msg); + releaseread_mrsw(&d->meta_lock); + return ret; +} - if ((is_worm_cart && (d->is_immutable || (d->is_appendonly && strcmp(name, "ltfs.vendor.IBM.immutable")))) - || (!is_worm_cart && (d->is_immutable || d->is_appendonly) && !_xattr_is_worm_ea(name))) { - /* EA cannot be set in case of immutable/appendonly */ - ltfsmsg(LTFS_ERR, 17237E, "set xattr: WORM entry"); - ret = -LTFS_RDONLY_XATTR; - goto out_unlock; +static int _xattr_get_string(const char *val, char **outval, const char *msg) +{ + if (! val) + return 0; + *outval = strdup(val); + if (! (*outval)) { + ltfsmsg(LTFS_ERR, 10001E, msg); + return -LTFS_NO_MEMORY; } + return 0; +} - /* Check if this is a user-writeable virtual xattr */ - if (_xattr_is_virtual(d, name, vol)) { - ret = _xattr_set_virtual(d, name, value, size, vol); - if (ret == -LTFS_NO_XATTR) - ret = -LTFS_RDONLY_XATTR; - goto out_unlock; +static int _xattr_get_u64(uint64_t val, char **outval, const char *msg) +{ + int ret = asprintf(outval, "%"PRIu64, val); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10001E, msg); + *outval = NULL; + ret = -LTFS_NO_MEMORY; } + return ret; +} - /* In the future, there could be user-writeable reserved xattrs. For now, just deny - * writes to all reserved xattrs not covered by the user-writeable virtual xattrs above. */ - if (strcasestr(name, "ltfs") == name && strcmp(name, "ltfs.spannedFileOffset") && strcmp(name, "ltfs.mediaPool.name") && - strcasestr(name, "ltfs.permissions.") != name && !_xattr_is_worm_ea(name)) { - ret = -LTFS_RDONLY_XATTR; - goto out_unlock; +static int _xattr_get_tapepos(struct tape_offset *val, char **outval, const char *msg) +{ + int ret = asprintf(outval, "%c:%"PRIu64, val->partition, val->block); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10001E, msg); + return -LTFS_NO_MEMORY; } + return 0; +} - acquirewrite_mrsw(&d->meta_lock); - - /* Search for existing xattr with this name. */ - ret = _xattr_seek(&xattr, d, name); +static int _xattr_get_partmap(struct ltfs_label *label, char **outval, const char *msg) +{ + int ret = asprintf(outval, "I:%c,D:%c", label->partid_ip, label->partid_dp); if (ret < 0) { - ltfsmsg(LTFS_ERR, 11122E, ret); - releasewrite_mrsw(&d->meta_lock); - goto out_unlock; - } - if (create && xattr) { - releasewrite_mrsw(&d->meta_lock); - ret = -LTFS_XATTR_EXISTS; - goto out_unlock; - } else if (replace && ! xattr) { - releasewrite_mrsw(&d->meta_lock); - ret = -LTFS_NO_XATTR; - goto out_unlock; + ltfsmsg(LTFS_ERR, 10001E, msg); + return -LTFS_NO_MEMORY; } - if (_xattr_is_worm_ea(name)) { - disable_worm_ea = (strncmp(value, "0", size) == 0); + return 0; +} - if (is_worm_cart && disable_worm_ea) { - ltfsmsg(LTFS_ERR, 17237E, "set xattr: clear WORM"); - releasewrite_mrsw(&d->meta_lock); - ret = -LTFS_XATTR_ERR; - goto out_unlock; - } - if (!disable_worm_ea) { - /* All values other than 0 is treated as 1 */ - value = new_value; - size = strlen(new_value); +static int _xattr_get_version(int version, char **outval, const char *msg) +{ + int ret; + if (version == 10000) { + *outval = strdup("1.0"); + if (! (*outval)) { + ltfsmsg(LTFS_ERR, 10001E, msg); + return -LTFS_NO_MEMORY; } - } - - if (!strcmp(name, "ltfs.mediaPool.name")) { - ret = tape_set_media_pool_info(vol, value, size, true); + } else { + ret = asprintf(outval, "%d.%d.%d", version/10000, (version % 10000)/100, version % 100); if (ret < 0) { - releasewrite_mrsw(&d->meta_lock); - goto out_unlock; + ltfsmsg(LTFS_ERR, 10001E, msg); + return -LTFS_NO_MEMORY; } - write_idx = true; } + return 0; +} - /* Set extended attribute */ - ret = xattr_do_set(d, name, value, size, xattr); - if (ret < 0) { - releasewrite_mrsw(&d->meta_lock); - goto out_unlock; - } +static int _xattr_set_time(struct dentry *d, struct ltfs_timespec *out, const char *value, + size_t size, const char *msg, struct ltfs_volume *vol) +{ + int ret; + struct ltfs_timespec t; + char *value_null_terminated; - /* update metadata */ - if (!strcmp(name, "ltfs.vendor.IBM.immutable")) { - d->is_immutable = !disable_worm_ea; - ltfsmsg(LTFS_INFO, 17238I, "immutable", d->is_immutable, d->name.name); - } - else if (!strcmp(name, "ltfs.vendor.IBM.appendonly")) { - d->is_appendonly = !disable_worm_ea; - ltfsmsg(LTFS_INFO, 17238I, "appendonly", d->is_appendonly, d->name.name); + value_null_terminated = malloc(size + 1); + if (! value_null_terminated) { + ltfsmsg(LTFS_ERR, 10001E, msg); + return -LTFS_NO_MEMORY; } + memcpy(value_null_terminated, value, size); + value_null_terminated[size] = '\0'; - get_current_timespec(&d->change_time); - releasewrite_mrsw(&d->meta_lock); - d->dirty = true; - ltfs_set_index_dirty(true, false, vol->index); - - if (write_idx) - ret = ltfs_sync_index(SYNC_EA, false, vol); - else - ret = 0; + ret = xml_parse_time(false, value_null_terminated, &t); + free(value_null_terminated); + if (ret < 0) + return -LTFS_BAD_ARG; -out_unlock: - _xattr_unlock_dentry(name, true, d, vol); + acquirewrite_mrsw(&d->meta_lock); + *out = t; + d->dirty = true; + releasewrite_mrsw(&d->meta_lock); + + ltfs_set_index_dirty(true, false, vol->index); return ret; } -/** - * Get an extended attribute. Returns an error if the provided buffer is not large enough - * to contain the attribute value. - * @param d File/directory to check - * @param name Xattr name - * @param value On success, contains xattr value - * @param size Output buffer size in bytes - * @param vol LTFS volume - * @return if size is nonzero, number of bytes returned in the value buffer. if size is zero, - * number of bytes in the xattr value. returns a negative value on error. If the - * operation needs to be restarted, then -LTFS_RESTART_OPERATION is returned instead. - */ -int xattr_get(struct dentry *d, const char *name, char *value, size_t size, - struct ltfs_volume *vol) +static int _xattr_get_vendorunique_xattr(char **outval, const char *msg, struct ltfs_volume *vol) { - struct xattr_info *xattr = NULL; int ret; - CHECK_ARG_NULL(d, -LTFS_NULL_ARG); - CHECK_ARG_NULL(name, -LTFS_NULL_ARG); - CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); - if (size > 0 && ! value) { - ltfsmsg(LTFS_ERR, 11123E); - return -LTFS_BAD_ARG; - } - - ret = _xattr_lock_dentry(name, false, d, vol); - if (ret < 0) - return ret; - - /* Try to get a virtual xattr first. */ - if (_xattr_is_virtual(d, name, vol)) { - ret = _xattr_get_virtual(d, value, size, name, vol); - if (ret == -LTFS_DEVICE_FENCED) { - _xattr_unlock_dentry(name, false, d, vol); - ret = ltfs_wait_revalidation(vol); - return (ret == 0) ? -LTFS_RESTART_OPERATION : ret; - } else if (NEED_REVAL(ret)) { - _xattr_unlock_dentry(name, false, d, vol); - ret = ltfs_revalidate(false, vol); - return (ret == 0) ? -LTFS_RESTART_OPERATION : ret; - } else if (IS_UNEXPECTED_MOVE(ret)) { - vol->reval = -LTFS_REVAL_FAILED; - _xattr_unlock_dentry(name, false, d, vol); - return ret; - }else if (ret != -LTFS_NO_XATTR) { - /* if ltfs.sync is specified, don't print any message */ - if (ret < 0 && ret != -LTFS_RDONLY_XATTR) - ltfsmsg(LTFS_ERR, 11128E, ret); - goto out_unlock; - } - } - - acquireread_mrsw(&d->meta_lock); + ret = ltfs_get_vendorunique_xattr(msg, outval, vol); + if (ret != 0) + *outval = NULL; - /* Look for a real xattr. */ - ret = _xattr_seek(&xattr, d, name); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 11129E, ret); - releaseread_mrsw(&d->meta_lock); - goto out_unlock; - } + return ret; +} - /* Generate output. */ - ret = 0; - if (! xattr) { - /* There's no such extended attribute */ - ret = -LTFS_NO_XATTR; - } else if (size && xattr->size > size) { - /* There is no space to fill the buffer */ - ret = -LTFS_SMALL_BUFFER; - } else if (size) { - /* Copy the extended attribute to the requester */ - memcpy(value, xattr->value, xattr->size); - ret = xattr->size; - } else /* size is zero */ { - /* Return how many bytes will be necessary to read this xattr */ - ret = xattr->size; - } +static int _xattr_set_vendorunique_xattr(const char *name, const char *value, size_t size, + struct ltfs_volume *vol) +{ + int ret; - releaseread_mrsw(&d->meta_lock); + ret = ltfs_set_vendorunique_xattr(name, value, size, vol); -out_unlock: - _xattr_unlock_dentry(name, false, d, vol); return ret; } /** - * Copy a list of extended attribute names to a user-provided buffer. - * @param d File/directory to get the list of extended attributes from - * @param list Output buffer for xattr names - * @param size Output buffer size in bytes - * @param vol LTFS volume - * @return number of bytes in buffer on success, or a negative value on error. + * Check xattr name is WORM related one or not + * + * @param name EA name to check + * @return true if name is related WORM. Otherwise false. */ -int xattr_list(struct dentry *d, char *list, size_t size, struct ltfs_volume *vol) +static inline bool _xattr_is_worm_ea(const char *name) { - int ret, nbytes = 0; - - CHECK_ARG_NULL(d, -LTFS_NULL_ARG); - CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); - if (size > 0 && ! list) { - ltfsmsg(LTFS_ERR, 11130E); - return -LTFS_BAD_ARG; + if (!strcmp(name, "ltfs.vendor.IBM.immutable") || !strcmp(name, "ltfs.vendor.IBM.appendonly")) { + /* WORM related xattr */ + return true; } + return false; +} - acquireread_mrsw(&d->meta_lock); - - /* Fill the buffer with only real xattrs. */ - if (size) - memset(list, 0, size); - - ret = _xattr_list_physicals(d, list, size); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 11133E, ret); - goto out; +/** + * Check xattr name is stored VEA or not + * + * Extended attribute starts from 'ltfs.' is reserved and treated as Virtual Extended Attribute (VEA) + * and they aren't stored into index. But 'ltfs.permissions.*' and 'ltfs.hash.*' is introduced from + * LTFS Format Spec 2.4 to store values into index. + * This function checks provided name shall be stored VEA or not. + * + * @param name EA name to check + * @return true if name is stored VEA. Otherwise false. + */ +static inline bool _xattr_is_stored_vea(const char *name) +{ + if (strcmp(name, "ltfs.spannedFileOffset") && + strcmp(name, "ltfs.mediaPool.name") && + strcasestr(name, "ltfs.permissions.") != name && + strcasestr(name, "ltfs.hash.") != name) + { + return false; } - nbytes += ret; - - /* - * There used to be an _xattr_list_virtuals function which was called here. - * Listing virtual xattrs causes problems with files copied from LTFS to another filesystem - * which are attempted to be brought back. Since the copy utility may also copy the - * reserved virtual extended attributes, the copy operation will fail with permission - * denied problems. - */ - - /* Was the buffer large enough? */ - if (size && (size_t)nbytes > size) - ret = -LTFS_SMALL_BUFFER; -out: - releaseread_mrsw(&d->meta_lock); - if (ret < 0) - return ret; - return nbytes; + return true; } +/* Local functions */ + /** - * Actually remove an extended attribute. - * @param dentry dentry to operate on - * @param name xattr name to delete - * @param force true to force removal, false to verify namespaces first - * @param vol LTFS volume - * @return 0 on success or a negative value on error + * Search for an xattr with the given name. Must call this function with a lock on + * the dentry's meta_lock. + * @param out On success, points to the xattr that was found. + * @param d Dentry to search. + * @param name Name to search for. + * @return 1 if xattr was found, 0 if not, or a negative value on error. */ -int xattr_do_remove(struct dentry *d, const char *name, bool force, struct ltfs_volume *vol) +static int _xattr_seek(struct xattr_info **out, struct dentry *d, const char *name) { - int ret; - struct xattr_info *xattr; - - acquirewrite_mrsw(&d->meta_lock); - - /* Look for a real extended attribute. */ - ret = _xattr_seek(&xattr, d, name); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 11140E, ret); - releasewrite_mrsw(&d->meta_lock); - return ret; - } else if (! xattr) { - releasewrite_mrsw(&d->meta_lock); - return -LTFS_NO_XATTR; - } + struct xattr_info *entry; - if (! force) { - /* If this xattr is in the reserved namespace, the user can't remove it. */ - /* TODO: in the future, there could be user-removable reserved xattrs. */ - if (strcasestr(name, "ltfs") == name && strcmp(name, "ltfs.spannedFileOffset") && - strcasestr(name, "ltfs.permissions.") != name && !_xattr_is_worm_ea(name) ) { - releasewrite_mrsw(&d->meta_lock); - return -LTFS_RDONLY_XATTR; + *out = NULL; + TAILQ_FOREACH(entry, &d->xattrlist, list) { + if (! strcmp(entry->key.name, name)) { + *out = entry; + break; } } - /* Remove the xattr. */ - TAILQ_REMOVE(&d->xattrlist, xattr, list); - get_current_timespec(&d->change_time); - releasewrite_mrsw(&d->meta_lock); + if (*out) + return 1; + else + return 0; +} - free(xattr->key.name); - if (xattr->value) - free(xattr->value); - free(xattr); +/** + * Take the volume lock and dentry contents_lock as appropriate for the EA and the access type. + * @param name EA name being read or written. + * @param modify True if EA will be set or deleted, false if it will be read. + * @param d Dentry under consideration. + * @param vol LTFS volume. + * @return 0 on success or -LTFS_REVAL_FAILED if the medium is invalid. + */ +static int _xattr_lock_dentry(const char *name, bool modify, struct dentry *d, struct ltfs_volume *vol) +{ + /* EAs that read the extent list need to take the contents_lock */ + if (! strcmp(name, "ltfs.startblock") + || ! strcmp(name, "ltfs.partition")) { + acquireread_mrsw(&d->contents_lock); + } + /* Other EAs either need no additional locks, or they need the meta_lock. + * The caller is responsible for taking the meta_lock as necessary. */ return 0; } /** - * Remove an extended attribute. - * @param d File/directory to operate on - * @param name Extended attribute name to delete - * @param vol LTFS volume - * @return 0 on success or a negative value on error + * Undo locking performed in _xattr_lock_dentry. + * @param name EA name being read or written. + * @param modify True if EA was set or deleted, false if it was read. + * @param d Dentry under consideration. + * @param vol LTFS volume. */ -int xattr_remove(struct dentry *d, const char *name, struct ltfs_volume *vol) +static void _xattr_unlock_dentry(const char *name, bool modify, struct dentry *d, struct ltfs_volume *vol) { - int ret; - bool is_worm_cart = false; - - CHECK_ARG_NULL(d, -LTFS_NULL_ARG); - CHECK_ARG_NULL(name, -LTFS_NULL_ARG); - CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); - - ret = _xattr_lock_dentry(name, true, d, vol); - if (ret < 0) - return ret; - - ret = tape_get_worm_status(vol->device, &is_worm_cart); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 17237E, "remove xattr: cart stat"); - ret = -LTFS_XATTR_ERR; - goto out_dunlk; + /* EAs that read the extent list need to take the contents_lock */ + if (! strcmp(name, "ltfs.startblock") + || ! strcmp(name, "ltfs.partition")) { + releaseread_mrsw(&d->contents_lock); } - - if ((d->is_immutable || d->is_appendonly) - && (is_worm_cart || !_xattr_is_worm_ea(name))) { - /* EA cannot be removed in case of immutable/appendonly */ - ltfsmsg(LTFS_ERR, 17237E, "remove xattr: WORM entry"); - ret = -LTFS_RDONLY_XATTR; - goto out_dunlk; - } - - /* If this xattr is virtual, try the virtual removal function. */ - if (_xattr_is_virtual(d, name, vol)) { - ret = _xattr_remove_virtual(d, name, vol); - if (ret == -LTFS_NO_XATTR) - ret = -LTFS_RDONLY_XATTR; /* non-removable virtual xattr */ - goto out_dunlk; - } - - ret = xattr_do_remove(d, name, false, vol); - if (ret < 0) - goto out_dunlk; - - if (!strcmp(name, "ltfs.vendor.IBM.immutable")) { - d->is_immutable = false; - ltfsmsg(LTFS_INFO, 17238I, "immutable", d->is_immutable, d->name.name); - } - else if (!strcmp(name, "ltfs.vendor.IBM.appendonly")) { - d->is_appendonly = false; - ltfsmsg(LTFS_INFO, 17238I, "appendonly", d->is_appendonly, d->name.name); - } - - d->dirty = true; - ltfs_set_index_dirty(true, false, vol->index); - -out_dunlk: - _xattr_unlock_dentry(name, true, d, vol); - return ret; -} - - -/** - * set LTFS_LIVELINK_EA_NAME - * @param path file path - * @param d File operate on - * @param vol LTFS volume - * @return 0 on success or a negative value on error - */ -int xattr_set_mountpoint_length(struct dentry *d, const char* value, size_t size ) -{ -#ifdef POSIXLINK_ONLY - return 0; -#else - int ret=0; - struct xattr_info *xattr; - - CHECK_ARG_NULL(d, -LTFS_NULL_ARG); - CHECK_ARG_NULL(value, -LTFS_NULL_ARG); - - acquireread_mrsw(&d->meta_lock); - ret = _xattr_seek(&xattr, d, LTFS_LIVELINK_EA_NAME); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 11129E, ret); - releaseread_mrsw(&d->meta_lock); - goto out_set; - } - ret = xattr_do_set(d, LTFS_LIVELINK_EA_NAME, value, size, xattr); - releaseread_mrsw(&d->meta_lock); - -out_set: - return ret; -#endif -} - - -/** - * Search for an xattr with the given name. Must call this function with a lock on - * the dentry's meta_lock. - * @param out On success, points to the xattr that was found. - * @param d Dentry to search. - * @param name Name to search for. - * @return 1 if xattr was found, 0 if not, or a negative value on error. - */ -int _xattr_seek(struct xattr_info **out, struct dentry *d, const char *name) -{ - struct xattr_info *entry; - - *out = NULL; - TAILQ_FOREACH(entry, &d->xattrlist, list) { - if (! strcmp(entry->key.name, name)) { - *out = entry; - break; - } - } - - if (*out) - return 1; - else - return 0; -} - -/** - * Take the volume lock and dentry contents_lock as appropriate for the EA and the access type. - * @param name EA name being read or written. - * @param modify True if EA will be set or deleted, false if it will be read. - * @param d Dentry under consideration. - * @param vol LTFS volume. - * @return 0 on success or -LTFS_REVAL_FAILED if the medium is invalid. - */ -int _xattr_lock_dentry(const char *name, bool modify, struct dentry *d, struct ltfs_volume *vol) -{ - /* EAs that read the extent list need to take the contents_lock */ - if (! strcmp(name, "ltfs.startblock") - || ! strcmp(name, "ltfs.partition")) { - acquireread_mrsw(&d->contents_lock); - } - - /* Other EAs either need no additional locks, or they need the meta_lock. - * The caller is responsible for taking the meta_lock as necessary. */ - return 0; -} - -/** - * Undo locking performed in _xattr_lock_dentry. - * @param name EA name being read or written. - * @param modify True if EA was set or deleted, false if it was read. - * @param d Dentry under consideration. - * @param vol LTFS volume. - */ -void _xattr_unlock_dentry(const char *name, bool modify, struct dentry *d, struct ltfs_volume *vol) -{ - /* EAs that read the extent list need to take the contents_lock */ - if (! strcmp(name, "ltfs.startblock") - || ! strcmp(name, "ltfs.partition")) { - releaseread_mrsw(&d->contents_lock); - } -} - -/** - * Strip a Linux namespace prefix from the given xattr name and return the position of the suffix. - * If the name is "user.X", return the "X" portion. Otherwise, return an error. - * This function does nothing on Mac OS X. - * @param name Name to strip. - * @return A pointer to the name suffix, or NULL to indicate an invalid name. On Mac OS X, - * always returns @name. - */ -const char *_xattr_strip_name(const char *name) -{ -#if (defined (__APPLE__) || defined (mingw_PLATFORM)) - return name; -#else - if (strstr(name, "user.") == name) - return name + 5; - else - return NULL; -#endif -} +} /** * List real extended attributes for a dentry. Must be called with a read lock held on @@ -665,7 +372,7 @@ const char *_xattr_strip_name(const char *name) * @param size Output buffer size, may be 0. * @return Number of bytes in listed extended attributes, or a negative value on error. */ -int _xattr_list_physicals(struct dentry *d, char *list, size_t size) +static int _xattr_list_physicals(struct dentry *d, char *list, size_t size) { struct xattr_info *entry; char *prefix = "\0", *new_name; @@ -718,7 +425,7 @@ int _xattr_list_physicals(struct dentry *d, char *list, size_t size) * @param vol LTFS volume to which the dentry belongs. * @return true if the name exists and is virtual, false otherwise. */ -bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_volume *vol) +static bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_volume *vol) { /* xattrs on all dentries */ if (! strcmp(name, "ltfs.createTime") @@ -826,8 +533,8 @@ bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_volume *v * -LTFS_NO_XATTR if no such readable virtual xattr exists, * -LTFS_RDONLY_XATTR for write-only virtual EAs, or another negative value on error. */ -int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, const char *name, - struct ltfs_volume *vol) +static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, const char *name, + struct ltfs_volume *vol) { int ret = -LTFS_NO_XATTR; char *val = NULL; @@ -1209,8 +916,8 @@ int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, const char * @return 0 on success, -LTFS_NO_XATTR if the xattr is not a settable virtual xattr, * or another negative value on error. */ -int _xattr_set_virtual(struct dentry *d, const char *name, const char *value, - size_t size, struct ltfs_volume *vol) +static int _xattr_set_virtual(struct dentry *d, const char *name, const char *value, + size_t size, struct ltfs_volume *vol) { int ret = 0; @@ -1556,7 +1263,7 @@ int _xattr_set_virtual(struct dentry *d, const char *name, const char *value, * @return 0 on success, -LTFS_NO_XATTR if the xattr is not a removable virtual xattr, or another * negative value on error. */ -int _xattr_remove_virtual(struct dentry *d, const char *name, struct ltfs_volume *vol) +static int _xattr_remove_virtual(struct dentry *d, const char *name, struct ltfs_volume *vol) { int ret = 0; @@ -1586,201 +1293,496 @@ int _xattr_remove_virtual(struct dentry *d, const char *name, struct ltfs_volume return ret; } -int _xattr_get_cartridge_health(cartridge_health_info *h, int64_t *val, char **outval, - const char *msg, struct ltfs_volume *vol) +/* Global functions */ + +int xattr_do_set(struct dentry *d, const char *name, const char *value, size_t size, + struct xattr_info *xattr) { - int ret = ltfs_get_cartridge_health(h, vol); - if (ret == 0) { - ret = asprintf(outval, "%"PRId64, *val); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - *outval = NULL; + int ret = 0; + + /* clear existing xattr or set up new one */ + if (xattr) { + if (xattr->value) { + free(xattr->value); + xattr->value = NULL; + } + } else { + xattr = (struct xattr_info *) calloc(1, sizeof(struct xattr_info)); + if (! xattr) { + ltfsmsg(LTFS_ERR, 10001E, "xattr_do_set: xattr"); return -LTFS_NO_MEMORY; } - } else - *outval = NULL; - return ret; -} - -int _xattr_get_cartridge_health_u64(cartridge_health_info *h, uint64_t *val, char **outval, - const char *msg, struct ltfs_volume *vol) -{ - int ret = ltfs_get_cartridge_health(h, vol); - if (ret == 0 && (int64_t)(*val) != UNSUPPORTED_CARTRIDGE_HEALTH) { - ret = asprintf(outval, "%"PRIu64, *val); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - *outval = NULL; + xattr->key.name = strdup(name); + if (! xattr->key.name) { + ltfsmsg(LTFS_ERR, 10001E, "xattr_do_set: xattr key"); ret = -LTFS_NO_MEMORY; + goto out_free; } - } else if (ret == 0) { - ret = asprintf(outval, "%"PRId64, UNSUPPORTED_CARTRIDGE_HEALTH); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - *outval = NULL; + xattr->key.percent_encode = fs_is_percent_encode_required(xattr->key.name); + TAILQ_INSERT_HEAD(&d->xattrlist, xattr, list); + } + + /* copy new value */ + xattr->size = size; + if (size > 0) { + xattr->value = (char *)malloc(size); + if (! xattr->value) { + ltfsmsg(LTFS_ERR, 10001E, "xattr_do_set: xattr value"); ret = -LTFS_NO_MEMORY; + goto out_remove; } - } else - *outval = NULL; + memcpy(xattr->value, value, size); + } + return 0; + +out_remove: + TAILQ_REMOVE(&d->xattrlist, xattr, list); +out_free: + if (xattr->key.name) + free(xattr->key.name); + free(xattr); return ret; } -int _xattr_get_cartridge_capacity(struct device_capacity *cap, unsigned long *val, char **outval, - const char *msg, struct ltfs_volume *vol) +/** + * Set an extended attribute. + * @param d File or directory to set the xattr on. + * @param name Name to set. + * @param value Value to set, may be binary, not necessarily null-terminated. + * @param size Size of value in bytes. + * @param flags XATTR_REPLACE to fail if xattr doesn't exist, XATTR_CREATE to fail if it does + * exist, or 0 to ignore any existing value. + * @return 0 on success or a negative value on error. + */ +int xattr_set(struct dentry *d, const char *name, const char *value, size_t size, + int flags, struct ltfs_volume *vol) { - double scale = vol->label->blocksize / 1048576.0; - int ret = ltfs_capacity_data_unlocked(cap, vol); - if (ret == 0) { - ret = asprintf(outval, "%lu", (unsigned long)((*val) * scale)); + struct xattr_info *xattr; + bool replace, create; + int ret; + bool is_worm_cart = false; + bool disable_worm_ea = false; + char *new_value="1"; + bool write_idx = false; + + CHECK_ARG_NULL(d, -LTFS_NULL_ARG); + CHECK_ARG_NULL(name, -LTFS_NULL_ARG); + CHECK_ARG_NULL(value, -LTFS_NULL_ARG); + CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); + if (size > LTFS_MAX_XATTR_SIZE) + return -LTFS_LARGE_XATTR; /* this is the error returned by ext3 when the xattr is too large */ + + replace = flags & XATTR_REPLACE; + create = flags & XATTR_CREATE; + + ret = _xattr_lock_dentry(name, true, d, vol); + if (ret < 0) + return ret; + + ret = tape_get_worm_status(vol->device, &is_worm_cart); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17237E, "set xattr: cart stat"); + ret = -LTFS_XATTR_ERR; + goto out_unlock; + } + + if ((is_worm_cart && (d->is_immutable || (d->is_appendonly && strcmp(name, "ltfs.vendor.IBM.immutable")))) + || (!is_worm_cart && (d->is_immutable || d->is_appendonly) && !_xattr_is_worm_ea(name))) { + /* EA cannot be set in case of immutable/appendonly */ + ltfsmsg(LTFS_ERR, 17237E, "set xattr: WORM entry"); + ret = -LTFS_RDONLY_XATTR; + goto out_unlock; + } + + /* Check if this is a user-writeable virtual xattr */ + if (_xattr_is_virtual(d, name, vol)) { + ret = _xattr_set_virtual(d, name, value, size, vol); + if (ret == -LTFS_NO_XATTR) + ret = -LTFS_RDONLY_XATTR; + goto out_unlock; + } + + /* In the future, there could be user-writeable reserved xattrs. For now, just deny + * writes to all reserved xattrs not covered by the user-writeable virtual xattrs above. */ + if (strcasestr(name, "ltfs") == name && !_xattr_is_stored_vea(name) && !_xattr_is_worm_ea(name)) { + ret = -LTFS_RDONLY_XATTR; + goto out_unlock; + } + + acquirewrite_mrsw(&d->meta_lock); + + /* Search for existing xattr with this name. */ + ret = _xattr_seek(&xattr, d, name); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 11122E, ret); + releasewrite_mrsw(&d->meta_lock); + goto out_unlock; + } + if (create && xattr) { + releasewrite_mrsw(&d->meta_lock); + ret = -LTFS_XATTR_EXISTS; + goto out_unlock; + } else if (replace && ! xattr) { + releasewrite_mrsw(&d->meta_lock); + ret = -LTFS_NO_XATTR; + goto out_unlock; + } + if (_xattr_is_worm_ea(name)) { + disable_worm_ea = (strncmp(value, "0", size) == 0); + + if (is_worm_cart && disable_worm_ea) { + ltfsmsg(LTFS_ERR, 17237E, "set xattr: clear WORM"); + releasewrite_mrsw(&d->meta_lock); + ret = -LTFS_XATTR_ERR; + goto out_unlock; + } + if (!disable_worm_ea) { + /* All values other than 0 is treated as 1 */ + value = new_value; + size = strlen(new_value); + } + } + + if (!strcmp(name, "ltfs.mediaPool.name")) { + ret = tape_set_media_pool_info(vol, value, size, true); if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - *outval = NULL; - return -LTFS_NO_MEMORY; + releasewrite_mrsw(&d->meta_lock); + goto out_unlock; } - } else - *outval = NULL; + write_idx = true; + } + + /* Set extended attribute */ + ret = xattr_do_set(d, name, value, size, xattr); + if (ret < 0) { + releasewrite_mrsw(&d->meta_lock); + goto out_unlock; + } + + /* update metadata */ + if (!strcmp(name, "ltfs.vendor.IBM.immutable")) { + d->is_immutable = !disable_worm_ea; + ltfsmsg(LTFS_INFO, 17238I, "immutable", d->is_immutable, d->name.name); + } + else if (!strcmp(name, "ltfs.vendor.IBM.appendonly")) { + d->is_appendonly = !disable_worm_ea; + ltfsmsg(LTFS_INFO, 17238I, "appendonly", d->is_appendonly, d->name.name); + } + + get_current_timespec(&d->change_time); + releasewrite_mrsw(&d->meta_lock); + d->dirty = true; + ltfs_set_index_dirty(true, false, vol->index); + + if (write_idx) + ret = ltfs_sync_index(SYNC_EA, false, vol); + else + ret = 0; + +out_unlock: + _xattr_unlock_dentry(name, true, d, vol); return ret; } -int _xattr_get_time(struct ltfs_timespec *val, char **outval, const char *msg) +/** + * Get an extended attribute. Returns an error if the provided buffer is not large enough + * to contain the attribute value. + * @param d File/directory to check + * @param name Xattr name + * @param value On success, contains xattr value + * @param size Output buffer size in bytes + * @param vol LTFS volume + * @return if size is nonzero, number of bytes returned in the value buffer. if size is zero, + * number of bytes in the xattr value. returns a negative value on error. If the + * operation needs to be restarted, then -LTFS_RESTART_OPERATION is returned instead. + */ +int xattr_get(struct dentry *d, const char *name, char *value, size_t size, + struct ltfs_volume *vol) { + struct xattr_info *xattr = NULL; int ret; - ret = xml_format_time(*val, outval); - if (! (*outval)) { - ltfsmsg(LTFS_ERR, 11145E, msg); - return -LTFS_NO_MEMORY; + CHECK_ARG_NULL(d, -LTFS_NULL_ARG); + CHECK_ARG_NULL(name, -LTFS_NULL_ARG); + CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); + if (size > 0 && ! value) { + ltfsmsg(LTFS_ERR, 11123E); + return -LTFS_BAD_ARG; } - return ret; -} + ret = _xattr_lock_dentry(name, false, d, vol); + if (ret < 0) + return ret; + + /* Try to get a virtual xattr first. */ + if (_xattr_is_virtual(d, name, vol)) { + ret = _xattr_get_virtual(d, value, size, name, vol); + if (ret == -LTFS_DEVICE_FENCED) { + _xattr_unlock_dentry(name, false, d, vol); + ret = ltfs_wait_revalidation(vol); + return (ret == 0) ? -LTFS_RESTART_OPERATION : ret; + } else if (NEED_REVAL(ret)) { + _xattr_unlock_dentry(name, false, d, vol); + ret = ltfs_revalidate(false, vol); + return (ret == 0) ? -LTFS_RESTART_OPERATION : ret; + } else if (IS_UNEXPECTED_MOVE(ret)) { + vol->reval = -LTFS_REVAL_FAILED; + _xattr_unlock_dentry(name, false, d, vol); + return ret; + }else if (ret != -LTFS_NO_XATTR) { + /* if ltfs.sync is specified, don't print any message */ + if (ret < 0 && ret != -LTFS_RDONLY_XATTR) + ltfsmsg(LTFS_ERR, 11128E, ret); + goto out_unlock; + } + } + + acquireread_mrsw(&d->meta_lock); + + /* Look for a real xattr. */ + ret = _xattr_seek(&xattr, d, name); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 11129E, ret); + releaseread_mrsw(&d->meta_lock); + goto out_unlock; + } + + /* Generate output. */ + ret = 0; + if (! xattr) { + /* There's no such extended attribute */ + ret = -LTFS_NO_XATTR; + } else if (size && xattr->size > size) { + /* There is no space to fill the buffer */ + ret = -LTFS_SMALL_BUFFER; + } else if (size) { + /* Copy the extended attribute to the requester */ + memcpy(value, xattr->value, xattr->size); + ret = xattr->size; + } else /* size is zero */ { + /* Return how many bytes will be necessary to read this xattr */ + ret = xattr->size; + } -int _xattr_get_dentry_time(struct dentry *d, struct ltfs_timespec *val, char **outval, - const char *msg) -{ - int ret; - acquireread_mrsw(&d->meta_lock); - ret = _xattr_get_time(val, outval, msg); releaseread_mrsw(&d->meta_lock); + +out_unlock: + _xattr_unlock_dentry(name, false, d, vol); return ret; } -int _xattr_get_string(const char *val, char **outval, const char *msg) +/** + * Copy a list of extended attribute names to a user-provided buffer. + * @param d File/directory to get the list of extended attributes from + * @param list Output buffer for xattr names + * @param size Output buffer size in bytes + * @param vol LTFS volume + * @return number of bytes in buffer on success, or a negative value on error. + */ +int xattr_list(struct dentry *d, char *list, size_t size, struct ltfs_volume *vol) { - if (! val) - return 0; - *outval = strdup(val); - if (! (*outval)) { - ltfsmsg(LTFS_ERR, 10001E, msg); - return -LTFS_NO_MEMORY; - } - return 0; -} + int ret, nbytes = 0; -int _xattr_get_u64(uint64_t val, char **outval, const char *msg) -{ - int ret = asprintf(outval, "%"PRIu64, val); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - *outval = NULL; - ret = -LTFS_NO_MEMORY; + CHECK_ARG_NULL(d, -LTFS_NULL_ARG); + CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); + if (size > 0 && ! list) { + ltfsmsg(LTFS_ERR, 11130E); + return -LTFS_BAD_ARG; } - return ret; -} -int _xattr_get_tapepos(struct tape_offset *val, char **outval, const char *msg) -{ - int ret = asprintf(outval, "%c:%"PRIu64, val->partition, val->block); + acquireread_mrsw(&d->meta_lock); + + /* Fill the buffer with only real xattrs. */ + if (size) + memset(list, 0, size); + + ret = _xattr_list_physicals(d, list, size); if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - return -LTFS_NO_MEMORY; + ltfsmsg(LTFS_ERR, 11133E, ret); + goto out; } - return 0; + nbytes += ret; + + /* + * There used to be an _xattr_list_virtuals function which was called here. + * Listing virtual xattrs causes problems with files copied from LTFS to another filesystem + * which are attempted to be brought back. Since the copy utility may also copy the + * reserved virtual extended attributes, the copy operation will fail with permission + * denied problems. + */ + + /* Was the buffer large enough? */ + if (size && (size_t)nbytes > size) + ret = -LTFS_SMALL_BUFFER; + +out: + releaseread_mrsw(&d->meta_lock); + if (ret < 0) + return ret; + return nbytes; } -int _xattr_get_partmap(struct ltfs_label *label, char **outval, const char *msg) +/** + * Actually remove an extended attribute. + * @param dentry dentry to operate on + * @param name xattr name to delete + * @param force true to force removal, false to verify namespaces first + * @param vol LTFS volume + * @return 0 on success or a negative value on error + */ +int xattr_do_remove(struct dentry *d, const char *name, bool force, struct ltfs_volume *vol) { - int ret = asprintf(outval, "I:%c,D:%c", label->partid_ip, label->partid_dp); + int ret; + struct xattr_info *xattr; + + acquirewrite_mrsw(&d->meta_lock); + + /* Look for a real extended attribute. */ + ret = _xattr_seek(&xattr, d, name); if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - return -LTFS_NO_MEMORY; + ltfsmsg(LTFS_ERR, 11140E, ret); + releasewrite_mrsw(&d->meta_lock); + return ret; + } else if (! xattr) { + releasewrite_mrsw(&d->meta_lock); + return -LTFS_NO_XATTR; } - return 0; -} -int _xattr_get_version(int version, char **outval, const char *msg) -{ - int ret; - if (version == 10000) { - *outval = strdup("1.0"); - if (! (*outval)) { - ltfsmsg(LTFS_ERR, 10001E, msg); - return -LTFS_NO_MEMORY; - } - } else { - ret = asprintf(outval, "%d.%d.%d", version/10000, (version % 10000)/100, version % 100); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - return -LTFS_NO_MEMORY; + if (! force) { + /* If this xattr is in the reserved namespace, the user can't remove it. */ + /* TODO: in the future, there could be user-removable reserved xattrs. */ + if (strcasestr(name, "ltfs") == name && !_xattr_is_stored_vea(name) && !_xattr_is_worm_ea(name)) { + releasewrite_mrsw(&d->meta_lock); + return -LTFS_RDONLY_XATTR; } } + + /* Remove the xattr. */ + TAILQ_REMOVE(&d->xattrlist, xattr, list); + get_current_timespec(&d->change_time); + releasewrite_mrsw(&d->meta_lock); + + free(xattr->key.name); + if (xattr->value) + free(xattr->value); + free(xattr); + return 0; } -int _xattr_set_time(struct dentry *d, struct ltfs_timespec *out, const char *value, size_t size, - const char *msg, struct ltfs_volume *vol) +/** + * Remove an extended attribute. + * @param d File/directory to operate on + * @param name Extended attribute name to delete + * @param vol LTFS volume + * @return 0 on success or a negative value on error + */ +int xattr_remove(struct dentry *d, const char *name, struct ltfs_volume *vol) { int ret; - struct ltfs_timespec t; - char *value_null_terminated; + bool is_worm_cart = false; - value_null_terminated = malloc(size + 1); - if (! value_null_terminated) { - ltfsmsg(LTFS_ERR, 10001E, msg); - return -LTFS_NO_MEMORY; + CHECK_ARG_NULL(d, -LTFS_NULL_ARG); + CHECK_ARG_NULL(name, -LTFS_NULL_ARG); + CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); + + ret = _xattr_lock_dentry(name, true, d, vol); + if (ret < 0) + return ret; + + ret = tape_get_worm_status(vol->device, &is_worm_cart); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17237E, "remove xattr: cart stat"); + ret = -LTFS_XATTR_ERR; + goto out_dunlk; } - memcpy(value_null_terminated, value, size); - value_null_terminated[size] = '\0'; - ret = xml_parse_time(false, value_null_terminated, &t); - free(value_null_terminated); + if ((d->is_immutable || d->is_appendonly) + && (is_worm_cart || !_xattr_is_worm_ea(name))) { + /* EA cannot be removed in case of immutable/appendonly */ + ltfsmsg(LTFS_ERR, 17237E, "remove xattr: WORM entry"); + ret = -LTFS_RDONLY_XATTR; + goto out_dunlk; + } + + /* If this xattr is virtual, try the virtual removal function. */ + if (_xattr_is_virtual(d, name, vol)) { + ret = _xattr_remove_virtual(d, name, vol); + if (ret == -LTFS_NO_XATTR) + ret = -LTFS_RDONLY_XATTR; /* non-removable virtual xattr */ + goto out_dunlk; + } + + ret = xattr_do_remove(d, name, false, vol); if (ret < 0) - return -LTFS_BAD_ARG; + goto out_dunlk; - acquirewrite_mrsw(&d->meta_lock); - *out = t; - d->dirty = true; - releasewrite_mrsw(&d->meta_lock); + if (!strcmp(name, "ltfs.vendor.IBM.immutable")) { + d->is_immutable = false; + ltfsmsg(LTFS_INFO, 17238I, "immutable", d->is_immutable, d->name.name); + } + else if (!strcmp(name, "ltfs.vendor.IBM.appendonly")) { + d->is_appendonly = false; + ltfsmsg(LTFS_INFO, 17238I, "appendonly", d->is_appendonly, d->name.name); + } + d->dirty = true; ltfs_set_index_dirty(true, false, vol->index); + +out_dunlk: + _xattr_unlock_dentry(name, true, d, vol); return ret; } -int _xattr_get_vendorunique_xattr(char **outval, const char *msg, struct ltfs_volume *vol) +/** + * Strip a Linux namespace prefix from the given xattr name and return the position of the suffix. + * If the name is "user.X", return the "X" portion. Otherwise, return an error. + * This function does nothing on Mac OS X. + * @param name Name to strip. + * @return A pointer to the name suffix, or NULL to indicate an invalid name. On Mac OS X, + * always returns @name. + */ +const char *xattr_strip_name(const char *name) { - int ret; - - ret = ltfs_get_vendorunique_xattr(msg, outval, vol); - if (ret != 0) - *outval = NULL; - - return ret; +#if (defined (__APPLE__) || defined (mingw_PLATFORM)) + return name; +#else + if (strstr(name, "user.") == name) + return name + 5; + else + return NULL; +#endif } -int _xattr_set_vendorunique_xattr(const char *name, const char *value, size_t size, struct ltfs_volume *vol) +/** + * set LTFS_LIVELINK_EA_NAME + * @param path file path + * @param d File operate on + * @param vol LTFS volume + * @return 0 on success or a negative value on error + */ +int xattr_set_mountpoint_length(struct dentry *d, const char* value, size_t size ) { - int ret; +#ifdef POSIXLINK_ONLY + return 0; +#else + int ret=0; + struct xattr_info *xattr; - ret = ltfs_set_vendorunique_xattr(name, value, size, vol); + CHECK_ARG_NULL(d, -LTFS_NULL_ARG); + CHECK_ARG_NULL(value, -LTFS_NULL_ARG); + + acquireread_mrsw(&d->meta_lock); + ret = _xattr_seek(&xattr, d, LTFS_LIVELINK_EA_NAME); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 11129E, ret); + releaseread_mrsw(&d->meta_lock); + goto out_set; + } + ret = xattr_do_set(d, LTFS_LIVELINK_EA_NAME, value, size, xattr); + releaseread_mrsw(&d->meta_lock); +out_set: return ret; +#endif } - -bool _xattr_is_worm_ea(const char *name) -{ - if (!strcmp(name, "ltfs.vendor.IBM.immutable") || !strcmp(name, "ltfs.vendor.IBM.appendonly")) { - /* WORM related xattr */ - return true; - } - return false; -} \ No newline at end of file diff --git a/src/libltfs/xattr.h b/src/libltfs/xattr.h index 030fdc6d..2a47568d 100644 --- a/src/libltfs/xattr.h +++ b/src/libltfs/xattr.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2021 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -48,6 +48,10 @@ ** IBM Almaden Research Center ** lucasvr@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -73,14 +77,11 @@ int xattr_get(struct dentry *d, const char *name, char *value, size_t size, int xattr_list(struct dentry *d, char *list, size_t size, struct ltfs_volume *vol); int xattr_remove(struct dentry *d, const char *name, struct ltfs_volume *vol); -int _xattr_get_string(const char *val, char **outval, const char *msg); -int _xattr_get_u64(uint64_t val, char **outval, const char *msg); - /** For internal use only */ int xattr_do_set(struct dentry *d, const char *name, const char *value, size_t size, struct xattr_info *xattr); int xattr_do_remove(struct dentry *d, const char *name, bool force, struct ltfs_volume *vol); -const char *_xattr_strip_name(const char *name); +const char *xattr_strip_name(const char *name); int xattr_set_mountpoint_length(struct dentry *d, const char* value, size_t size); #ifdef __cplusplus From 171312641c5b2b0ab7af5f044e8dbf9a54cc40f5 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 17 Dec 2021 17:19:11 +0900 Subject: [PATCH 069/121] Fix a few vulnerabilities (#323) --- messages/libltfs/root.txt | 2 +- messages/tape_linux_lin_tape/root.txt | 2 +- src/kmi/flatfile.c | 2 -- src/libltfs/arch/time_internal.h | 4 ++-- src/libltfs/index_criteria.c | 10 ++++++++-- src/libltfs/ltfstrace.c | 16 ++++++++-------- src/libltfs/ltfstrace.h | 8 ++++---- .../linux/lin_tape/lin_tape_ibmtape.c | 2 +- 8 files changed, 25 insertions(+), 21 deletions(-) diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index c36e1f77..9b5eea35 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -237,7 +237,7 @@ root:table { 11140E:string { "Cannot remove extended attribute: failed to look up the xattr (%d)." } 11141E:string { "Cannot list physical xattrs: failed to generate namespace prefix (%d)." } 11142E:string { "Cannot list physical xattrs: failed to convert key to system locale (%d)." } - //unused 11143E + 11143E:string { "Specified length of buffer is too short. (%lu)" } //unused 11144E:string { "Cannot list physical xattrs: failed to convert key to system locale (%d)." } 11145E:string { "Cannot get attribute %s: failed to generate the time string." } 11146E:string { "Invalid index criteria option \'%s\'." } diff --git a/messages/tape_linux_lin_tape/root.txt b/messages/tape_linux_lin_tape/root.txt index 7845c4a6..780aa3e7 100644 --- a/messages/tape_linux_lin_tape/root.txt +++ b/messages/tape_linux_lin_tape/root.txt @@ -68,7 +68,7 @@ root:table { 30423I:string { "Opening a device through ibmtape driver (%s)." } 30424E:string { "%s: medium is already mounted or in use." } 30425I:string { "Cannot open device \'%s\' (%d)." } - 30426W:string { "Device \'%s\' must be opened in read-only mode." } + 30426W:string { "Device \'%s\' must be opened in read-write mode." } 30427I:string { "Cannot open device: inquiry failed (%d)." } 30428I:string { "Product ID is \'%s\'." } 30429I:string { "Vendor ID is %s." } diff --git a/src/kmi/flatfile.c b/src/kmi/flatfile.c index 628e9c27..53735d49 100644 --- a/src/kmi/flatfile.c +++ b/src/kmi/flatfile.c @@ -198,7 +198,6 @@ int flatfile_get_key(unsigned char **keyalias, unsigned char **key, void * const if (ret < 0) { ltfsmsg(LTFS_ERR, 15552E); if (dk_list) { - memset(dk_list, 0, strlen((char *) dk_list)); free(dk_list); } return ret; @@ -210,7 +209,6 @@ int flatfile_get_key(unsigned char **keyalias, unsigned char **key, void * const /* * Cache DK and DKi for revalidation at tape drive POR * if (dk_list) { - * memset(dk_list, 0, strlen((char *) dk_list)); * free(dk_list); * dk_list = NULL; * } diff --git a/src/libltfs/arch/time_internal.h b/src/libltfs/arch/time_internal.h index b84d1929..71508d76 100644 --- a/src/libltfs/arch/time_internal.h +++ b/src/libltfs/arch/time_internal.h @@ -76,12 +76,12 @@ struct ltfs_timespec { #define TIMER_TYPE_OSX (0x0000000000000001) #define TIMER_TYPE_WINDOWS (0x0000000000000002) -#pragma pack(1) +#pragma pack(push, 1) struct timer_info { uint64_t type; uint64_t base; }; -#pragma pack(0) +#pragma pack(pop) #define LTFS_TIME_T_MAX (253402300799) /* 9999/12/31 23:59:59 UTC */ #define LTFS_TIME_T_MIN (-62167219200) /* 0000/01/01 00:00:00 UTC */ diff --git a/src/libltfs/index_criteria.c b/src/libltfs/index_criteria.c index c6c9ecff..928d029b 100644 --- a/src/libltfs/index_criteria.c +++ b/src/libltfs/index_criteria.c @@ -199,10 +199,16 @@ bool index_criteria_find_option(const char *str, const char *substr, */ int index_criteria_parse_size(const char *criteria, size_t len, struct index_criteria *ic) { - int ret = 0, multiplier = 1; + int ret = 0, multiplier = 1, sizelen = 0; char rule[len+1], last, *ptr; - snprintf(rule, len-strlen("size="), "%s", criteria + strlen("size=")); + sizelen = strlen("size="); + if ((len - sizelen) < 0) { + ltfsmsg(LTFS_ERR, 11143E, len); + return -LTFS_POLICY_INVALID; + } + + snprintf(rule, len - sizelen, "%s", criteria + sizelen); for (ptr=&rule[0]; *ptr; ptr++) { if (isalpha(*ptr) && *(ptr+1) && isalpha(*(ptr+1))) { diff --git a/src/libltfs/ltfstrace.c b/src/libltfs/ltfstrace.c index b02dd239..7efa73b9 100644 --- a/src/libltfs/ltfstrace.c +++ b/src/libltfs/ltfstrace.c @@ -97,7 +97,7 @@ * Definition for LTFS trace header information */ #define LTFS_TRACE_SIGNATURE "LTFS_TRC" -#pragma pack(1) +#pragma pack(push, 1) struct trace_header { char signature[8]; /**< Signature for LTFS trace */ uint32_t header_size; /**< Size of trace header */ @@ -108,12 +108,12 @@ struct trace_header { uint32_t trace_size; /**< Whole size of trace (all headers and bodies) */ uint32_t crc; /**< CRC (reserved for future use) */ }; -#pragma pack(0) +#pragma pack(pop) /* * Definitions for LTFS Request header information */ -#pragma pack(1) +#pragma pack(push, 1) struct request_header { uint32_t header_size; /**< Size of request header */ uint32_t num_of_req_trace; /**< Number of request trace descriptrs (always 1) */ @@ -123,12 +123,12 @@ struct request_header { } req_t_desc; /**< Request header descriptor */ uint32_t crc; /**< CRC (reserved for future use) */ }; -#pragma pack(0) +#pragma pack(pop) /* * Definitions for LTFS function trace header */ -#pragma pack(1) +#pragma pack(push, 1) struct function_trace_header { uint32_t header_size; /**< Size of function trace header */ uint32_t num_of_fn_trace; /**< Number of function trace */ @@ -139,19 +139,19 @@ struct function_trace_header { } *req_t_desc; uint32_t crc; /**< CRC (reserved for future use) */ }; -#pragma pack(0) +#pragma pack(pop) /* * Definitions for LTFS function trace data */ -#pragma pack(1) +#pragma pack(push, 1) struct function_entry { uint64_t time; uint64_t function; uint64_t info1; uint64_t info2; }; -#pragma pack(0) +#pragma pack(pop) #define FN_TRACE_ENTRY_SIZE (sizeof(struct function_entry)) diff --git a/src/libltfs/ltfstrace.h b/src/libltfs/ltfstrace.h index a44b0047..d336fa40 100644 --- a/src/libltfs/ltfstrace.h +++ b/src/libltfs/ltfstrace.h @@ -175,18 +175,18 @@ int ltfs_request_profiler_stop(void); /* * Definitions for LTFS request trace */ -#pragma pack(1) +#pragma pack(push, 1) struct profiler_entry { uint64_t time; uint32_t req_num; uint32_t tid; }; -#pragma pack(0) +#pragma pack(pop) #define PROF_ENTRY_SIZE (sizeof(struct profiler_entry)) #define REQ_PROF_ENTRY_SIZE PROF_ENTRY_SIZE /* Don't record information fields in profler data */ -#pragma pack(1) +#pragma pack(push, 1) struct request_entry { uint64_t time; uint32_t req_num; @@ -194,7 +194,7 @@ struct request_entry { uint64_t info1; uint64_t info2; }; -#pragma pack(0) +#pragma pack(pop) #define REQ_TRACE_ENTRY_SIZE (sizeof(struct request_entry)) #define REQ_TRACE_SIZE (4 * 1024 * 1024) /* 4MB */ diff --git a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c index b9efb827..146d8dc1 100644 --- a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c +++ b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c @@ -1056,7 +1056,7 @@ int lin_tape_ibmtape_open(const char *devname, void **handle) priv->fd = open(devfile, O_RDWR | O_NDELAY); if (priv->fd < 0) { - priv->fd = open(devname, O_RDONLY | O_NDELAY); + priv->fd = open(devfile, O_RDONLY | O_NDELAY); if (priv->fd < 0) { if (errno == EAGAIN) { ltfsmsg(LTFS_ERR, 30424E, devname); From 93de9d88031603d6843268b28ff706a2c4946ab6 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 17 Dec 2021 21:30:06 +0900 Subject: [PATCH 070/121] Fix a potential buffer overflow (#324) --- src/libltfs/index_criteria.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/libltfs/index_criteria.c b/src/libltfs/index_criteria.c index 928d029b..88819b16 100644 --- a/src/libltfs/index_criteria.c +++ b/src/libltfs/index_criteria.c @@ -199,15 +199,17 @@ bool index_criteria_find_option(const char *str, const char *substr, */ int index_criteria_parse_size(const char *criteria, size_t len, struct index_criteria *ic) { - int ret = 0, multiplier = 1, sizelen = 0; + int ret = 0, multiplier = 1; + size_t sizelen = 0; char rule[len+1], last, *ptr; sizelen = strlen("size="); - if ((len - sizelen) < 0) { + if (len <= sizelen) { ltfsmsg(LTFS_ERR, 11143E, len); return -LTFS_POLICY_INVALID; } + memset(rule, 0, sizeof(rule)); snprintf(rule, len - sizelen, "%s", criteria + sizelen); for (ptr=&rule[0]; *ptr; ptr++) { From ae184803c56f50752d99f812cef3c3aaf867c7bc Mon Sep 17 00:00:00 2001 From: piste2750 <32239919+piste2750@users.noreply.github.com> Date: Fri, 24 Dec 2021 15:50:51 +0900 Subject: [PATCH 071/121] Fix build break on FreeBSD (#326) --- src/tape_drivers/freebsd/cam/cam_tc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 6fdb2582..5ae6ee9b 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -283,7 +283,7 @@ int camtape_open(const char *devname, void **handle) while(*cur) { if ((! strncmp((char*)softc->cd->inq_data.vendor, (*cur)->vendor_id, strlen((*cur)->vendor_id)) ) && (! strncmp((char*)softc->cd->inq_data.product, (*cur)->product_id, strlen((*cur)->product_id)) ) ) { - priv->vendor = (*cur)->vendor_type; + softc->vendor = (*cur)->vendor_type; drive_type = (*cur)->drive_type; break; } From 3f2e5c64510c130c275d452c32719ce644499e0d Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Thu, 13 Jan 2022 12:24:16 +0900 Subject: [PATCH 072/121] Fix link failure in Debian11 (#329) --- configure.ac | 1 + 1 file changed, 1 insertion(+) diff --git a/configure.ac b/configure.ac index 649da0c6..da2d0d04 100644 --- a/configure.ac +++ b/configure.ac @@ -572,6 +572,7 @@ fi if test "x${host_mac}" = "xno" then MODULE_CHECK_LD='-Wl,--no-undefined,--as-needed' + AM_LDFLAGS="${AM_LDFLAGS} -ldl" fi AM_CFLAGS=`echo ${AM_CFLAGS} | ${SED} 's|-D_FORTIFY_SOURCE=. ||g'` From 42c08710bda914e2bf0ac3749c69d830e9b45577 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 26 Jan 2022 14:46:21 +0900 Subject: [PATCH 073/121] Introduce Index mount mode (#330) Introduce a feature to mount LTFS with an index file. The feature includes 1. Mount a tape with an index file and tape, contents of tape can be accessible on read-only 2. Mount a tape only with index, meta-data in the index file can be accessible In addition, a few functions and improvements are introduced too. - Change -o rollback_mount option in ltfs to have a generation number or an index file to mount - Change --capture-index option in ltfs and ltfsck to have a destination directory - Improve the performance of capturing index --- man/ltfs.8 | 15 +- man/ltfsck.8 | 9 +- man/sgml/ltfs.sgml | 15 +- man/sgml/ltfsck.sgml | 7 +- messages/bin_ltfs/root.txt | 16 +- messages/bin_ltfsck/root.txt | 8 +- messages/libltfs/root.txt | 5 +- src/libltfs/ltfs.c | 154 ++++++++--- src/libltfs/ltfs.h | 32 ++- src/libltfs/ltfs_fsops.c | 6 +- src/libltfs/ltfs_internal.c | 59 +++- src/libltfs/ltfs_internal.h | 9 +- src/libltfs/periodic_sync.c | 9 +- src/libltfs/xattr.c | 8 +- src/libltfs/xml.h | 6 +- src/libltfs/xml_libltfs.h | 10 +- src/libltfs/xml_reader.c | 22 +- src/libltfs/xml_reader_libltfs.c | 58 ++-- src/libltfs/xml_writer.c | 15 +- src/libltfs/xml_writer_libltfs.c | 43 ++- src/ltfs_copyright.h | 22 +- src/ltfs_fuse.c | 5 +- src/ltfs_fuse.h | 8 +- src/main.c | 450 +++++++++++++++++++------------ src/utils/ltfsck.c | 122 ++++++--- 25 files changed, 740 insertions(+), 373 deletions(-) diff --git a/man/ltfs.8 b/man/ltfs.8 index e7e39331..1466fe0b 100644 --- a/man/ltfs.8 +++ b/man/ltfs.8 @@ -1,5 +1,5 @@ .\" auto-generated by docbook2man-spec from docbook-utils package -.TH "LTFS" "8" "19 August 2021" "LTFS" "LTFS Command Reference" +.TH "LTFS" "8" "20 January 2022" "LTFS" "LTFS Command Reference" .SH NAME ltfs \- File system based on a linear tape drive .SH SYNOPSIS @@ -70,8 +70,11 @@ Only use for a CM corrupted medium \fB-o device_list\fR Show available tape devices .TP -\fB-o rollback_mount=\fIgen\fB\fR -Attempt to mount on previous index generation (read-only mount) +\fB-o rollback_mount=\fIgen|index_file\fB\fR +Attempt to mount on previous index generation on tape or specified index file by read-only mode + +When both index_file and device name is specified on -o devname option, normal read-only mount is attempted. +When index_file is only specified, meta-data read-only mode is attempted. EAGAIN would be returned at accessing contents of file. .TP \fB-o release_device\fR Clear device reservation (should be specified with -o devname @@ -179,8 +182,10 @@ Override output verbosity directly (default: 2) \fB-o noeject\fR Do not eject the cartridge after unmount (default) .TP -\fB-o capture_index\fR -Capture latest index to work directory at unmount +\fB--capture-index=\fIdir\fB\fR +Capture index to the specified directory by dir when index is updated. +File name of each index is [BARCODE]-[GEN]-[PARTITION].xml if tape serial (barcode) +is specified at format time. Otherwise it is [VOL_UUID]-[GEN]-[PARTITION].xml. .TP \fB-o scsi_append_only_mode=\fIon|off\fB\fR Set the tape device append-only mode (default=on) diff --git a/man/ltfsck.8 b/man/ltfsck.8 index 378e6990..35e510ff 100644 --- a/man/ltfsck.8 +++ b/man/ltfsck.8 @@ -1,5 +1,5 @@ .\" auto-generated by docbook2man-spec from docbook-utils package -.TH "LTFSCK" "8" "19 August 2021" "LTFS" "LTFS Command Reference" +.TH "LTFSCK" "8" "20 January 2022" "LTFS" "LTFS Command Reference" .SH NAME ltfsck \- Recover and rollback utility for LTFS formatted tape .SH SYNOPSIS @@ -90,9 +90,10 @@ Use the specified key manager interface backend (default: none) \fB-x, --fulltrace\fR Enable full function call tracing (slow) .TP -\fB--capture-index\fR -Capture index information to the current directory -(-g is effective for this option) +\fB--capture-index=\fIdir\fB\fR +Capture indexes read successfully to the specified directory by dir. (-g is effective for this option) +File name of each index is [BARCODE]-[GEN]-[PARTITION].xml if tape serial (barcode) +is specified at format time. Otherwise it is [VOL_UUID]-[GEN]-[PARTITION].xml. .TP \fB--salvage-rollback-points\fR List the rollback points of the cartridge that has no EOD diff --git a/man/sgml/ltfs.sgml b/man/sgml/ltfs.sgml index 69d34b61..077dcd66 100644 --- a/man/sgml/ltfs.sgml +++ b/man/sgml/ltfs.sgml @@ -156,9 +156,12 @@ - + - Attempt to mount on previous index generation (read-only mount) + Attempt to mount on previous index generation on tape or specified index file by read-only mode + + When both index_file and device name is specified on -o devname option, normal read-only mount is attempted. + When index_file is only specified, meta-data read-only mode is attempted. EAGAIN would be returned at accessing contents of file. @@ -356,9 +359,13 @@ - + - Capture latest index to work directory at unmount + + Capture index to the specified directory by dir when index is updated. + File name of each index is [BARCODE]-[GEN]-[PARTITION].xml if tape serial (barcode) + is specified at format time. Otherwise it is [VOL_UUID]-[GEN]-[PARTITION].xml. + diff --git a/man/sgml/ltfsck.sgml b/man/sgml/ltfsck.sgml index 4a8d023f..0e9bcd08 100644 --- a/man/sgml/ltfsck.sgml +++ b/man/sgml/ltfsck.sgml @@ -209,11 +209,12 @@ M - + - Capture index information to the current directory - (-g is effective for this option) + Capture indexes read successfully to the specified directory by dir. (-g is effective for this option) + File name of each index is [BARCODE]-[GEN]-[PARTITION].xml if tape serial (barcode) + is specified at format time. Otherwise it is [VOL_UUID]-[GEN]-[PARTITION].xml. diff --git a/messages/bin_ltfs/root.txt b/messages/bin_ltfs/root.txt index 91d19bd7..5ff7b519 100644 --- a/messages/bin_ltfs/root.txt +++ b/messages/bin_ltfs/root.txt @@ -3,7 +3,7 @@ // OO_Copyright_BEGIN // // -// Copyright 2010, 2020 IBM Corp. All rights reserved. +// Copyright 2010, 2022 IBM Corp. All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions @@ -52,7 +52,7 @@ root:table { //unused 14010E:string { "Cannot load I/O scheduler \'%s\'." } 14011E:string { "Cannot allocate LTFS volume structure." } 14012E:string { "Tape backend option parsing failed." } - 14013E:string { "Cannot mount the volume." } + 14013E:string { "Cannot mount the volume from %s." } //unused 14014E:string { "Cannot duplicate index criteria." } 14015W:string { "Volume does not allow index criteria override. Ignoring user-specified criteria." } 14016E:string { "Cannot format data placement rules (%d)." } @@ -152,7 +152,13 @@ root:table { 14113I:string { "Specified mount point is listed if succeeded." } 14114E:string { "Cannot initialize the open file table." } 14115E:string { "Invalid scsi_append_only_mode option: %s." } - 14116E:string { "This medium is not supported (%d)." } + 14116E:string { "This medium is not supported (%d)." } + 14117I:string { "Metadata only rollback mount is specified. Mounting as meta read-only from %s." } + 14118E:string { "Unexpected option is specified for read only mount (%d)." } + 14119I:string { "Rollback mount from index file is specified. Mounting as read-only from %s." } + 14120W:string { "Cannot access to directory %s, disabled index capture mode (%d)." } + 14121I:string { "Index will be captured at %s at update" } + 14122I:string { "Index will not be captured." } // 14150 - 14199 are reserved for LE+ @@ -209,7 +215,7 @@ root:table { // Reserved 14434E // Reserved 14435I 14436I:string { " -o device_list Show available tape devices" } - 14437I:string { " -o rollback_mount= Attempt to mount on previous index generation (read-only mount)" } + 14437I:string { " -o rollback_mount= Attempt to mount on previous index generation on tape or specified index file (read-only mount)" } // Reserved 14438I 14439I:string { " -o noeject Do not eject the cartridge after unmount (default)" } 14440I:string { " -o noatime Do not update index if only access times have changed (default)" } @@ -227,7 +233,7 @@ root:table { // Reserved 14451I // Reserved 14454I 14455I:string { " -o kmi_backend= Key manager interface implementation to use (default: %s, use \"none\" to disable)" } - 14456I:string { " -o capture_index Capture latest index to work directory at unmount" } + 14456I:string { " -o capture_index= Capture index to the specified directory by dir when index is updated" } // Reserved 14457I // Reserved 14458I // Reserved 14459I diff --git a/messages/bin_ltfsck/root.txt b/messages/bin_ltfsck/root.txt index 16a6b4ee..4c5cc797 100644 --- a/messages/bin_ltfsck/root.txt +++ b/messages/bin_ltfsck/root.txt @@ -3,7 +3,7 @@ // OO_Copyright_BEGIN // // -// Copyright 2010, 2020 IBM Corp. All rights reserved. +// Copyright 2010, 2022 IBM Corp. All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions @@ -151,6 +151,10 @@ root:table { 16109E:string { "This operation is not allowed on this medium (%s)." } 16110E:string { "The --salvage-rollback-points option was specified against a normal cartridge." } 16111I:string { "The recovery process is skipped because of a locked cartridge (%d)." } + 16112W:string { "Cannot rename %s to %s (%d)." } + 16113W:string { "Cannot access to directory %s, disabled index capture mode (%d)." } + 16114I:string { "Index will be captured in %s." } + 16115I:string { "Index will not be captured." } // Help messages 16400I:string { "Usage: %s [options] filesys" } @@ -180,7 +184,7 @@ root:table { " (Must be used for a cartridge that cannot be recovered by a normal option.)" } 16422I:string { " -m, --full-index-info Display full index information (Effective only for -l option)" } 16423I:string { " --kmi-backend= Override the default key manager interface backend" } - 16424I:string { " --capture-index Capture index information to the current directory (-g is effective for this option)" } + 16424I:string { " --capture-index= Capture indexes to the specified directory (-g is effective for this option)" } 16425I:string { " --syslogtrace Enable diagnostic output to stderr and syslog" } 16426I:string { " -V, --version Version information" } 16427I:string { " --salvage-rollback-points List the rollback points of the cartridge that has no EOD" } diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index 9b5eea35..4a9f757f 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -3,7 +3,7 @@ // OO_Copyright_BEGIN // // -// Copyright 2010, 2020 IBM Corp. All rights reserved. +// Copyright 2010, 2022 IBM Corp. All rights reserved. // // Redistribution and use in source and binary forms, with or without // modification, are permitted provided that the following conditions @@ -97,7 +97,7 @@ root:table { 11002E:string { "Cannot instantiate LTFS volume: failed to allocate index data." } 11003E:string { "Cannot retrieve device capacity data (%d)." } 11004E:string { "Cannot take the device lock (%s)." } - 11005I:string { "Mounting the volume." } + 11005I:string { "Mounting the volume from %s." } 11006E:string { "Cannot read volume: failed to load the tape." } 11007D:string { "Tape is loaded." } 11008D:string { "Reading partition labels." } @@ -831,6 +831,7 @@ v 17287I:string { "Making R/O mount from the location (%c, %llu)." } 17288I:string { "Detected the final indexes (IP: Gen = %llu, Pos = %llu) (DP: Gen = %llu, Pos = %llu)." } 17289I:string { "Skip parsing the final index on IP." } + 17290E:string { "UUID in the index does not match the label." } // For Debug 19999I:string { "%s %s %d." } diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index dc215bc0..541a67cf 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -81,11 +81,6 @@ #define MIN(a, b) ((a) < (b) ? (a) : (b)) #endif -/* O_BINARY is defined only in MinGW */ -#ifndef O_BINARY -#define O_BINARY 0 -#endif - volatile char *copyright = LTFS_COPYRIGHT_0"\n"LTFS_COPYRIGHT_1"\n"LTFS_COPYRIGHT_2"\n" \ LTFS_COPYRIGHT_3"\n"LTFS_COPYRIGHT_4"\n"LTFS_COPYRIGHT_5"\n"; @@ -420,8 +415,10 @@ void _ltfs_volume_free(bool force, struct ltfs_volume **volume) free((*volume)->mountpoint); if ((*volume)->t_attr) free((*volume)->t_attr); - if ((*volume)->index_cache_path) - free((*volume)->index_cache_path); + if ((*volume)->index_cache_path_w) + free((*volume)->index_cache_path_w); + if ((*volume)->index_cache_path_r) + free((*volume)->index_cache_path_r); destroy_mrsw(&(*volume)->lock); ltfs_thread_mutex_destroy(&(*volume)->reval_lock); ltfs_thread_cond_destroy(&(*volume)->reval_cond); @@ -551,6 +548,13 @@ int ltfs_test_unit_ready(struct ltfs_volume *vol) ret = ltfs_get_volume_lock(false, vol); if (ret < 0) return ret; + + if (vol->mount_type == MOUNT_ROLLBACK_META) { + /* Return good when volume is meta-data mount mode */ + releaseread_mrsw(&vol->lock); + return 0; + } + ret = tape_device_lock(vol->device); if (ret == -LTFS_DEVICE_FENCED) { ret = ltfs_wait_revalidation(vol); @@ -647,6 +651,13 @@ int ltfs_capacity_data(struct device_capacity *cap, struct ltfs_volume *vol) ret = ltfs_get_volume_lock(false, vol); if (ret < 0) return ret; + + if (vol->mount_type == MOUNT_ROLLBACK_META) { + /* Return good when volume is meta-data mount mode */ + releaseread_mrsw(&vol->lock); + return 0; + } + ret = ltfs_capacity_data_unlocked(cap, vol); if (ret == -LTFS_DEVICE_FENCED) { ret = ltfs_wait_revalidation(vol); @@ -1531,7 +1542,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec char *vl_print = NULL; char *barcode = NULL; - ltfsmsg(LTFS_INFO, 11005I); + ltfsmsg(LTFS_INFO, 11005I, "device"); CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); @@ -1724,7 +1735,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec goto out_unlock; } else { INTERRUPTED_GOTO(ret, out_unlock); - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, false, vol); if (ret < 0) { if (read_ip) ltfsmsg(LTFS_ERR, 11024E); /* read IP Index failed */ @@ -1757,7 +1768,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec goto out_unlock; } else { INTERRUPTED_GOTO(ret, out_unlock); - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, false, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 11021E); /* read DP Index failed */ goto out_unlock; @@ -1786,7 +1797,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec goto out_unlock; } else { INTERRUPTED_GOTO(ret, out_unlock); - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, false, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 11024E); /* read IP Index failed */ goto out_unlock; @@ -1813,17 +1824,17 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec INTERRUPTED_GOTO(ret, out_unlock); if(gen != 0 && gen != vol->index->generation) { if(is_worm_recovery_mount){ - ret = ltfs_traverse_index_no_eod(vol, ltfs_ip_id(vol), gen, NULL, NULL, NULL); + ret = ltfs_traverse_index_no_eod(vol, ltfs_ip_id(vol), gen, false, NULL, NULL, NULL); if(ret < 0) - ret = ltfs_traverse_index_no_eod(vol, ltfs_dp_id(vol), gen, NULL, NULL, NULL); + ret = ltfs_traverse_index_no_eod(vol, ltfs_dp_id(vol), gen, false, NULL, NULL, NULL); } else if(vol->traverse_mode == TRAVERSE_FORWARD){ - ret = ltfs_traverse_index_forward(vol, ltfs_ip_id(vol), gen, NULL, NULL, NULL); + ret = ltfs_traverse_index_forward(vol, ltfs_ip_id(vol), gen, false, NULL, NULL, NULL); if(ret < 0) - ret = ltfs_traverse_index_forward(vol, ltfs_dp_id(vol), gen, NULL, NULL, NULL); + ret = ltfs_traverse_index_forward(vol, ltfs_dp_id(vol), gen, false, NULL, NULL, NULL); } else { - ret = ltfs_traverse_index_backward(vol, ltfs_ip_id(vol), gen, NULL, NULL, NULL); + ret = ltfs_traverse_index_backward(vol, ltfs_ip_id(vol), gen, false, NULL, NULL, NULL); if(ret < 0) - ret = ltfs_traverse_index_backward(vol, ltfs_dp_id(vol), gen, NULL, NULL, NULL); + ret = ltfs_traverse_index_backward(vol, ltfs_dp_id(vol), gen, false, NULL, NULL, NULL); } if (ret < 0) { ltfsmsg(LTFS_ERR, 17079E, gen); @@ -1904,6 +1915,62 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec return ret; } +/** + * Mount LTFS volume from provided index file + * Volume shall be mounted as read-only to avoid illegal data update. + * + * @param filename file name of the index + * @param label_check check label on tape (for R/O mount with data access) on true + * @param vol the volume to load + * @return 0 on success or a negative value on error. + */ +int ltfs_mount_indexfile(char* filename, bool label_check, struct ltfs_volume *vol) +{ + int ret = 0; + + ltfsmsg(LTFS_INFO, 11005I, filename); + + CHECK_ARG_NULL(filename, -LTFS_NULL_ARG); + CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); + + if (label_check) { + /* load tape, read indexes, set compression */ + ret = ltfs_start_mount(false, vol); + if (ret < 0) { + /* ltfs_start_mount() generated an appropriate error message */ + goto out_unlock; + } + ltfsmsg(LTFS_DEBUG, 11013D); /* partition labels are valid */ + vol->mount_type = MOUNT_ROLLBACK; + } else { + /* Assume 512KB block*/ + vol->label->blocksize = 512 * KB; + vol->mount_type = MOUNT_ROLLBACK_META; + } + + vol->first_locate.tv_sec = 0; + vol->first_locate.tv_nsec = 0; + + ret = ltfs_read_indexfile(filename, false, vol); + + if (label_check) { + if (strcmp(vol->index->vol_uuid, vol->label->vol_uuid)) { + /* + * Volume UUID in label and it on index is not matched. + * Actual UUID is not printed to avoid illegal modification by hand. + */ + ltfsmsg(LTFS_ERR, 17290E); + ret = -LTFS_LABEL_MISMATCH; + } + } + +out_unlock: + if (ret < 0 && vol->index) + ltfs_index_free(&vol->index); + + return ret; +} + /** * Load cartridge attribute varues from CM * @param vol the volume to get attribute @@ -2001,7 +2068,8 @@ int ltfs_unmount(char *reason, struct ltfs_volume *vol) start: ret = ltfs_get_volume_lock(true, vol); - if (!ret) { + if (!ret && vol->mount_type != MOUNT_ROLLBACK_META) { + ret = tape_get_cart_volume_lock_status(vol->device, &vollock); if (vol->mount_type == MOUNT_NORMAL && @@ -2427,12 +2495,12 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) (vol->ip_index_file_end && vol->index->selfptr.partition == ltfs_ip_id(vol)))) { /* Surpress on-disk index cache write on the recursive call */ - cache_path_save = vol->index_cache_path; - vol->index_cache_path = NULL; + cache_path_save = vol->index_cache_path_w; + vol->index_cache_path_w = NULL; ret = ltfs_write_index(ltfs_dp_id(vol), reason, vol); /* Restore cache path to handle on-disk index cache */ - vol->index_cache_path = cache_path_save; + vol->index_cache_path_w = cache_path_save; cache_path_save = NULL; if (NEED_REVAL(ret)) @@ -2625,11 +2693,11 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) * Write down the state of the current LTFS file system to a XML file on disk. * Acquires a write lock on vol->index->root->lock. * @param work_dir LTFS work directory. - * @param need_gen include generation number to file name + * @param id partition ID or 0 * @param vol LTFS volume * @return 0 on success or a negative value on error */ -int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, struct ltfs_volume *vol) +int ltfs_save_index_to_disk(const char *work_dir, char * reason, char id, struct ltfs_volume *vol) { char *path; int ret; @@ -2641,13 +2709,13 @@ int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, /* Write the schema to a file on disk */ ltfsmsg(LTFS_DEBUG, 17182D, vol->label->vol_uuid, vol->label->barcode); - if (need_gen) { - if (strcmp(vol->label->barcode, " ")) - ret = asprintf(&path, "%s/%s-%d.schema", work_dir, vol->label->barcode, vol->index->generation); + if (id) { + if (HAVE_BARCODE(vol)) + ret = asprintf(&path, "%s/%s-%d-%c.schema", work_dir, vol->label->barcode, vol->index->generation, id); else - ret = asprintf(&path, "%s/%s-%d.schema", work_dir, vol->label->vol_uuid, vol->index->generation); + ret = asprintf(&path, "%s/%s-%d-%c.schema", work_dir, vol->label->vol_uuid, vol->index->generation, id); } else { - if (strcmp(vol->label->barcode, " ")) + if (HAVE_BARCODE(vol)) ret = asprintf(&path, "%s/%s.schema", work_dir, vol->label->barcode); else ret = asprintf(&path, "%s/%s.schema", work_dir, vol->label->vol_uuid); @@ -2658,8 +2726,7 @@ int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, return -ENOMEM; } - - ltfsmsg(LTFS_INFO, 17235I, _get_barcode(vol), 'Z', "Volume Cache", + ltfsmsg(LTFS_INFO, 17235I, _get_barcode(vol), id ? id : 'Z', "Volume Cache", (unsigned long long)vol->index->file_count, path); ret = xml_schema_to_file(path, vol->index->creator, reason, vol->index); @@ -2671,7 +2738,7 @@ int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, } /* Change index file's mode */ - if (chmod(path, S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH)) { + if (chmod(path, S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP)) { ret = -errno; ltfsmsg(LTFS_ERR, 17184E, errno); } @@ -2679,7 +2746,7 @@ int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, ltfsmsg(LTFS_INFO, 17236I, _get_barcode(vol), (unsigned long long)vol->index->generation, - 'Z', + id ? id : 'Z', (unsigned long long)vol->index->selfptr.block, path); @@ -2777,7 +2844,7 @@ int ltfs_set_barcode(const char *barcode, struct ltfs_volume *vol) } strcpy(vol->label->barcode, barcode); } else - strcpy(vol->label->barcode, " "); + strcpy(vol->label->barcode, NO_BARCODE); return 0; } @@ -3583,13 +3650,14 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) * @param vol LTFS volume * @param partition partition to traverse * @param gen generation to search. 0 to search all + * @param skip_dir skip parsing directory * @param func call back function to call when an index is find. NULL not to call * @param list this pointer is specified as 3rd arguments of call back function * @param priv private data for call back function * @return 0 on success or a negative value on error */ int ltfs_traverse_index_no_eod(struct ltfs_volume *vol, char partition, unsigned int gen, - f_index_found func, void **list, void* priv) + bool skip_dir, f_index_found func, void **list, void* priv) { int ret, func_ret; @@ -3602,7 +3670,7 @@ int ltfs_traverse_index_no_eod(struct ltfs_volume *vol, char partition, unsigned while (true) { ltfs_index_free(&vol->index); ltfs_index_alloc(&vol->index, vol); - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, skip_dir, vol); if (ret < 0 && ret != -LTFS_UNSUPPORTED_INDEX_VERSION) { ltfsmsg(LTFS_ERR, 17075E, 'N', (int)vol->device->position.block, partition); return ret; @@ -3656,13 +3724,14 @@ int ltfs_traverse_index_no_eod(struct ltfs_volume *vol, char partition, unsigned * @param vol LTFS volume * @param partition partition to traverse * @param gen generation to search. 0 to search all + * @param skip_dir skip parsing directory * @param func call back function to call when an index is find. NULL not to call * @param list this pointer is specified as 3rd arguments of call back function * @param priv private data for call back function * @return 0 on success or a negative value on error */ int ltfs_traverse_index_forward(struct ltfs_volume *vol, char partition, unsigned int gen, - f_index_found func, void **list, void* priv) + bool skip_dir, f_index_found func, void **list, void* priv) { int ret, func_ret; struct tape_offset last_index; @@ -3685,7 +3754,7 @@ int ltfs_traverse_index_forward(struct ltfs_volume *vol, char partition, unsigne while (last_index.block >= vol->device->position.block) { ltfs_index_free(&vol->index); ltfs_index_alloc(&vol->index, vol); - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, skip_dir, vol); if (ret < 0 && ret != -LTFS_UNSUPPORTED_INDEX_VERSION) { ltfsmsg(LTFS_ERR, 17075E, 'F', (int)vol->device->position.block, partition); return ret; @@ -3740,12 +3809,13 @@ int ltfs_traverse_index_forward(struct ltfs_volume *vol, char partition, unsigne * @param vol LTFS volume * @param partition partition to traverse * @param gen generation to search. 0 to search all + * @param skip_dir skip parsing directory * @param func call back function to call when an index is find. NULL not to call * @param list this pointer is specified as 3rd arguments of call back function * @return 0 on success or a negative value on error */ int ltfs_traverse_index_backward(struct ltfs_volume *vol, char partition, unsigned int gen, - f_index_found func, void **list, void* priv) + bool skip_dir, f_index_found func, void **list, void* priv) { int ret, func_ret; @@ -3761,7 +3831,7 @@ int ltfs_traverse_index_backward(struct ltfs_volume *vol, char partition, unsign ltfs_index_free(&vol->index); ltfs_index_alloc(&vol->index, vol); - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, skip_dir, vol); if (ret < 0 && ret != -LTFS_UNSUPPORTED_INDEX_VERSION) { ltfsmsg(LTFS_ERR, 17075E, 'B', (int)vol->device->position.block, partition); return ret; @@ -3932,7 +4002,7 @@ static int _ltfs_detect_final_rec_dp(struct ltfs_volume *vol, struct tc_position INTERRUPTED_RETURN(); ltfsmsg(LTFS_INFO, 17120I, "DP", (unsigned long long)seekpos.partition, (unsigned long long)seekpos.block); - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, false, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 17121E, "DP", ret); return ret; @@ -4108,7 +4178,7 @@ int ltfs_recover_eod(struct ltfs_volume *vol) * In index partition, Index will be overwritten. * Read an index only current partition is DP. */ - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, false, vol); if (ret < 0) return ret; } diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index 34aabe40..07b006aa 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -48,6 +48,10 @@ ** IBM Almaden Research Center ** lucasvr@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ #ifndef __ltfs_h__ @@ -61,6 +65,10 @@ extern "C" { #include "arch/win/win_util.h" #endif +/* O_BINARY is defined only in MinGW */ +#ifndef O_BINARY +#define O_BINARY 0 +#endif #include #include @@ -200,6 +208,9 @@ struct device_data; do {sleep(1);}while(1); \ }while(0) +#define NO_BARCODE " " +#define HAVE_BARCODE(v) strcmp(v->label->barcode, NO_BARCODE) + /* Callback prototype used to list directories. The function must return 0 on success * or a negative value on error. */ typedef int (*ltfs_dir_filler) (void *buf, const char *name, void *priv); @@ -375,9 +386,10 @@ enum volumelock_status { }; enum volume_mount_type { - MOUNT_NORMAL = 0, /**< Normal mount */ - MOUNT_ROLLBACK, /**< Roll back mount */ - MOUNT_ERR_TAPE, /**< Mount write perm tape */ + MOUNT_NORMAL = 0, /**< Normal mount */ + MOUNT_ROLLBACK, /**< Roll back mount */ + MOUNT_ROLLBACK_META, /**< Roll back mount only with meta-data */ + MOUNT_ERR_TAPE, /**< Mount write perm tape */ }; #define VOL_WRITE_PERM_MASK (0xE0) @@ -393,7 +405,8 @@ struct ltfs_volume { struct tc_coherency dp_coh; /**< Data partition coherency info */ struct ltfs_label *label; /**< Information from the partition labels */ struct ltfs_index *index; /**< Current cartridge index */ - char *index_cache_path; /**< File name of on-disk index cache */ + char *index_cache_path_w; /**< File name of on-disk index cache update at writing an index */ + char *index_cache_path_r; /**< File name of on-disk index cache update at parsing an index */ /* Opaque handles to higher-level structures */ void *iosched_handle; /**< Handle to the I/O scheduler state */ @@ -642,6 +655,7 @@ bool ltfs_get_criteria_allow_update(struct ltfs_volume *vol); int ltfs_start_mount(bool trial, struct ltfs_volume *vol); int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool recover_symlink, unsigned short gen, struct ltfs_volume *vol); +int ltfs_mount_indexfile(char* filename, bool label_check, struct ltfs_volume *vol); int ltfs_unmount(char *reason, struct ltfs_volume *vol); void ltfs_dump_tree_unlocked(struct ltfs_index *index); void ltfs_dump_tree(struct ltfs_volume *vol); @@ -679,7 +693,7 @@ int ltfs_parse_library_backend_opts(void *opt_args, void *opts); void ltfs_set_index_dirty(bool locking, bool atime, struct ltfs_index *idx); void ltfs_unset_index_dirty(bool update_version, struct ltfs_index *idx); int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol); -int ltfs_save_index_to_disk(const char *work_dir, char * reason, bool need_gen, struct ltfs_volume *vol); +int ltfs_save_index_to_disk(const char *work_dir, char *reason, char id, struct ltfs_volume *vol); char ltfs_dp_id(struct ltfs_volume *vol); char ltfs_ip_id(struct ltfs_volume *vol); @@ -688,11 +702,11 @@ const char *ltfs_get_volume_uuid(struct ltfs_volume *vol); int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol); int ltfs_traverse_index_forward(struct ltfs_volume *vol, char partition, unsigned int gen, - f_index_found func, void **list, void *priv); + bool skip_dir, f_index_found func, void **list, void *priv); int ltfs_traverse_index_backward(struct ltfs_volume *vol, char partition, unsigned int gen, - f_index_found func, void **list, void *priv); + bool skip_dir, f_index_found func, void **list, void *priv); int ltfs_traverse_index_no_eod(struct ltfs_volume *vol, char partition, unsigned int gen, - f_index_found func, void **list, void *priv); + bool skip_dir, f_index_found func, void **list, void *priv); int ltfs_check_eod_status(struct ltfs_volume *vol); int ltfs_recover_eod(struct ltfs_volume *vol); int ltfs_release_medium(struct ltfs_volume *vol); diff --git a/src/libltfs/ltfs_fsops.c b/src/libltfs/ltfs_fsops.c index 89fa1e50..efd37869 100644 --- a/src/libltfs/ltfs_fsops.c +++ b/src/libltfs/ltfs_fsops.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2021 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -1798,6 +1798,10 @@ ssize_t ltfs_fsops_read(struct dentry *d, char *buf, size_t count, off_t offset, if (d->isdir) return -LTFS_ISDIRECTORY; + if (vol->mount_type == MOUNT_ROLLBACK_META) { + return -LTFS_DEVICE_UNREADY; + } + if (iosched_initialized(vol)) ret = iosched_read(d, buf, count, offset, vol); else diff --git a/src/libltfs/ltfs_internal.c b/src/libltfs/ltfs_internal.c index efdcbdf5..2f449f4e 100644 --- a/src/libltfs/ltfs_internal.c +++ b/src/libltfs/ltfs_internal.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -44,6 +44,10 @@ ** IBM Almaden Research Center ** bbiskebo@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -384,11 +388,13 @@ int ltfs_read_one_label(tape_partition_t partition, struct ltfs_label *label, * This function does not read over another file mark * @param eod_pos EOD position for current partition, or 0 to assume that EOD will not be * encountered during parsing. + * @param recover_symlink recover symlink conflict + * @param skip_dir skip parsing directory * @param vol the volume * @return 0 on success, 1 if index file does not end with a file mark (but is otherwise valid), * or a negative value on error. */ -int ltfs_read_index(uint64_t eod_pos, bool recover_symlink, struct ltfs_volume *vol) +int ltfs_read_index(uint64_t eod_pos, bool recover_symlink, bool skip_dir, struct ltfs_volume *vol) { int ret, ret_sym; struct tc_position pos; @@ -410,7 +416,7 @@ int ltfs_read_index(uint64_t eod_pos, bool recover_symlink, struct ltfs_volume * } /* Parse and validate the schema */ - ret = xml_schema_from_tape(eod_pos, vol); + ret = xml_schema_from_tape(eod_pos, skip_dir, vol); if ( vol->index->symerr_count ) { if ( recover_symlink ) { ret_sym = ltfs_split_symlink( vol ); @@ -473,6 +479,49 @@ int ltfs_read_index(uint64_t eod_pos, bool recover_symlink, struct ltfs_volume * return end_fm ? 0 : 1; } +/** + * Read an index file from file, storing the result in the given volume. + * + * @param filename file name of index + * @param recover_symlink recover symlink conflict + * @param vol the volume + * @return 0 on success, or a negative value on error. + */ +int ltfs_read_indexfile(char* filename, bool recover_symlink, struct ltfs_volume *vol) +{ + int ret, ret_sym; + + ltfs_index_free(&vol->index); + ret = ltfs_index_alloc(&vol->index, vol); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 11297E, ret); + return ret; + } + + /* Parse and validate the schema */ + ret = xml_schema_from_file(filename, vol->index, vol); + if ( vol->index->symerr_count ) { + if ( recover_symlink ) { + ret_sym = ltfs_split_symlink( vol ); + if (ret_sym < 0) { + ret = ret_sym; + } else if (ret == -LTFS_SYMLINK_CONFLICT) { + ret = 0; + } + } else { + ltfsmsg(LTFS_ERR, 11321E); + } + free( vol->index->symlink_conflict ); + vol->index->symerr_count = 0; + } + + if (ret < 0) { + ltfsmsg(LTFS_WARN, 11194W, ret); + } + + return ret; +} + /** * Returns true iff a given char corresponds to a valid logical partition ID. */ @@ -540,7 +589,7 @@ int ltfs_seek_index(char partition, tape_block_t *eod_pos, tape_block_t *index_e /* try to read an index file */ check_err(tape_spacefm(vol->device, 1), 11202E, out); - ret = ltfs_read_index(*eod_pos, recover_symlink, vol); + ret = ltfs_read_index(*eod_pos, recover_symlink, false, vol); if (ret < 0) { /* no index file found: go back 2 file marks and try again */ ltfsmsg(LTFS_DEBUG, 11204D); @@ -704,7 +753,7 @@ int _ltfs_check_pointers(struct ltfs_index *ip_index, struct ltfs_index *dp_inde ret = tape_seek(vol->device, &seekpos); if (ret < 0) return ret; - ret = ltfs_read_index(0, false, vol); + ret = ltfs_read_index(0, false, true, vol); if (ret < 0) return ret; dp_backptr = vol->index->backptr.block; diff --git a/src/libltfs/ltfs_internal.h b/src/libltfs/ltfs_internal.h index 2ce075c3..e02365ea 100644 --- a/src/libltfs/ltfs_internal.h +++ b/src/libltfs/ltfs_internal.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -44,6 +44,10 @@ ** IBM Almaden Research Center ** bbiskebo@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -70,7 +74,8 @@ int ltfs_check_medium(bool fix, bool deep, bool recover_extra, bool recover_syml int ltfs_read_labels(bool trial, struct ltfs_volume *vol); int ltfs_read_one_label(tape_partition_t partition, struct ltfs_label *label, struct ltfs_volume *vol); -int ltfs_read_index(uint64_t eod_pos, bool recover_symlink, struct ltfs_volume *vol); +int ltfs_read_index(uint64_t eod_pos, bool recover_symlink, bool skip_dir, struct ltfs_volume *vol); +int ltfs_read_indexfile(char* filename, bool recover_symlink, struct ltfs_volume *vol); int ltfs_update_cart_coherency(struct ltfs_volume *vol); int ltfs_write_index_conditional(char partition, struct ltfs_volume *vol); diff --git a/src/libltfs/periodic_sync.c b/src/libltfs/periodic_sync.c index 13571214..33d71ae6 100644 --- a/src/libltfs/periodic_sync.c +++ b/src/libltfs/periodic_sync.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -43,6 +43,7 @@ ** AUTHOR: Atsushi Abe ** IBM Yamato, Japan ** PISTE@jp.ibm.com +** ************************************************************************************* */ @@ -85,6 +86,12 @@ ltfs_thread_return periodic_sync_thread(void* data) if (! priv->keepalive) break; + if (priv->vol->mount_type == MOUNT_ROLLBACK || + priv->vol->mount_type == MOUNT_ROLLBACK_META) { + /* Never call sync on R/O mount */ + continue; + } + ltfs_request_trace(FUSE_REQ_ENTER(REQ_SYNC), 0, 0); ltfsmsg(LTFS_DEBUG, 17067D, "Sync-by-Time"); diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index bcf85b6b..3f188a95 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2021 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -1516,6 +1516,12 @@ int xattr_get(struct dentry *d, const char *name, char *value, size_t size, /* Try to get a virtual xattr first. */ if (_xattr_is_virtual(d, name, vol)) { + + if (vol->mount_type == MOUNT_ROLLBACK_META) { + _xattr_unlock_dentry(name, false, d, vol); + return -LTFS_DEVICE_UNREADY; + } + ret = _xattr_get_virtual(d, value, size, name, vol); if (ret == -LTFS_DEVICE_FENCED) { _xattr_unlock_dentry(name, false, d, vol); diff --git a/src/libltfs/xml.h b/src/libltfs/xml.h index b81726f9..4701242e 100644 --- a/src/libltfs/xml.h +++ b/src/libltfs/xml.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -50,7 +50,7 @@ ** ** Atsushi Abe ** IBM Tokyo Lab., Japan -** piste@jp.ibm.com +** piste@jp.ibm.com ** ************************************************************************************* */ @@ -280,6 +280,8 @@ struct xml_input_tape { uint64_t eod_pos; /**< EOD position of the current partition. */ bool saw_small_block; /**< Have we seen a small block yet? */ bool saw_file_mark; /**< If we saw a small blilock, was it a file mark? */ + int fd; /**< File Descriptor for index cache if fd > 0 */ + int errno_fd; /**< errno from the index cache */ char *buf; /**< 1-block input buffer. */ uint32_t buf_size; /**< Input buffer size. */ uint32_t buf_start; /**< Offset of first valid byte in input buffer. */ diff --git a/src/libltfs/xml_libltfs.h b/src/libltfs/xml_libltfs.h index b50a3800..2f64e98f 100644 --- a/src/libltfs/xml_libltfs.h +++ b/src/libltfs/xml_libltfs.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -36,7 +36,7 @@ ** ** COMPONENT NAME: IBM Linear Tape File System ** -** FILE NAME: xml.h +** FILE NAME: xml_libltfs.h ** ** DESCRIPTION: Prototypes for XML read/write functions. ** @@ -48,6 +48,10 @@ ** IBM Almaden Research Center ** lucasvr@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -85,7 +89,7 @@ int xml_schema_to_tape(char *reason, struct ltfs_volume *vol); int xml_label_from_file(const char *filename, struct ltfs_label *label); int xml_label_from_mem(const char *buf, int buf_size, struct ltfs_label *label); int xml_schema_from_file(const char *filename, struct ltfs_index *idx, struct ltfs_volume *vol); -int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol); +int xml_schema_from_tape(uint64_t eod_pos, bool skip_dir, struct ltfs_volume *vol); int xml_extent_symlink_info_from_file(const char *filename, struct dentry *d); #endif /* __xml_libltfs_h */ diff --git a/src/libltfs/xml_reader.c b/src/libltfs/xml_reader.c index 6087e592..80e029a4 100644 --- a/src/libltfs/xml_reader.c +++ b/src/libltfs/xml_reader.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -48,6 +48,10 @@ ** IBM Almaden Research Center ** lucasvr@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -507,7 +511,7 @@ int xml_input_tape_read_callback(void *context, char *buffer, int len) struct xml_input_tape *ctx = context; ssize_t nread, nr2; char *buf2; - int bytes_saved, bytes_remaining, ret_sp; + int bytes_saved, bytes_remaining, ret_sp, ret_fd; if (len == 0) return 0; @@ -541,8 +545,20 @@ int xml_input_tape_read_callback(void *context, char *buffer, int len) /* Try to read a block into the buffer. */ nread = tape_read(ctx->vol->device, ctx->buf, ctx->buf_size, false, - ctx->vol->kmi_handle); + ctx->vol->kmi_handle); ++ctx->current_pos; + + /* Write down the data read to the read cache */ + if (ctx->fd > 0 && nread > 0) { + ret_fd = write(ctx->fd, ctx->buf, nread); + if (ret_fd < 0) { + ltfsmsg(LTFS_ERR, 17244E, (int)errno); + ctx->errno_fd = -LTFS_CACHE_IO; + return -1; + } + } + + /* Condition check of data read */ if (nread < 0) { /* We know we're not at EOD, so read errors are unexpected. */ ltfsmsg(LTFS_ERR, 17039E, (int)nread); diff --git a/src/libltfs/xml_reader_libltfs.c b/src/libltfs/xml_reader_libltfs.c index 4935dcd4..a134c4a9 100644 --- a/src/libltfs/xml_reader_libltfs.c +++ b/src/libltfs/xml_reader_libltfs.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -1485,10 +1485,12 @@ static int _xml_parse_dirtree(xmlTextReaderPtr reader, struct dentry *parent, * with the nodes found during the scanning. * @param reader Source of XML data * @param idx LTFS index + * @param skip_dir skip parsing directory and file * @param vol LTFS volume to which the index belongs. May be NULL. * @return 0 on success or a negative value on error. */ -static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, struct ltfs_volume *vol) +static int _xml_parse_schema(xmlTextReaderPtr reader, bool skip_dir, + struct ltfs_index *idx, struct ltfs_volume *vol) { unsigned long long value_int; declare_parser_vars("ltfsindex"); @@ -1584,9 +1586,13 @@ static int _xml_parse_schema(xmlTextReaderPtr reader, struct ltfs_index *idx, st } else if (! strcmp(name, "directory")) { check_required_tag(6); assert_not_empty(); - ret = _xml_parse_dirtree(reader, NULL, idx, vol, NULL); - if (ret < 0) - return ret; + if (skip_dir) { + xml_skip_tag(reader); + } else { + ret = _xml_parse_dirtree(reader, NULL, idx, vol, NULL); + if (ret < 0) + return ret; + } } else if (! strcmp(name, "previousgenerationlocation")) { check_optional_tag(0); @@ -1824,7 +1830,7 @@ int xml_label_from_mem(const char *buf, int buf_size, struct ltfs_label *label) * with the nodes found during the scanning. * @param filename XML input file. * @param idx LTFS index. - * @param vol LTFS volume to which the index belongs. May be NULL. + * @param vol LTFS volume to which the index belongs. Can be NULL. * @return 0 on success or a negative value on error. */ int xml_schema_from_file(const char *filename, struct ltfs_index *idx, struct ltfs_volume *vol) @@ -1846,7 +1852,7 @@ int xml_schema_from_file(const char *filename, struct ltfs_index *idx, struct lt * unknown tags modifies the behavior of xmlFreeTextReader so that an additional * xmlDocFree call is required to free all memory. */ doc = xmlTextReaderCurrentDoc(reader); - ret = _xml_parse_schema(reader, idx, vol); + ret = _xml_parse_schema(reader, false, idx, vol); if (ret < 0) ltfsmsg(LTFS_ERR, 17012E, filename, ret); if (doc) @@ -1869,13 +1875,14 @@ int xml_schema_from_file(const char *filename, struct ltfs_index *idx, struct lt * the file mark. * @param eod_pos EOD block position for the current partition, or 0 to assume EOD will not be * encountered during parsing. + * @param skip_dir skip parsing directory * @param vol LTFS volume. * @return 0 on success, 1 if parsing succeeded but no file mark was encountered, * or a negative value on error. */ -int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) +int xml_schema_from_tape(uint64_t eod_pos, bool skip_dir, struct ltfs_volume *vol) { - int ret; + int ret, bk = -1; struct tc_position current_pos; struct xml_input_tape *ctx; xmlParserInputBufferPtr read_buf; @@ -1902,15 +1909,21 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) free(ctx); return -LTFS_NO_MEMORY; } - ctx->vol = vol; - ctx->err_code = 0; - ctx->current_pos = current_pos.block; - ctx->eod_pos = eod_pos; + + ctx->fd = -1; + ctx->errno_fd = 0; + if (vol->index_cache_path_r) + xml_acquire_file_lock(vol->index_cache_path_r, &ctx->fd, &bk, true); + + ctx->vol = vol; + ctx->err_code = 0; + ctx->current_pos = current_pos.block; + ctx->eod_pos = eod_pos; ctx->saw_small_block = false; - ctx->saw_file_mark = false; - ctx->buf_size = vol->label->blocksize; - ctx->buf_start = 0; - ctx->buf_used = 0; + ctx->saw_file_mark = false; + ctx->buf_size = vol->label->blocksize; + ctx->buf_start = 0; + ctx->buf_used = 0; /* Create input buffer pointer. */ read_buf = xmlParserInputBufferCreateIO(xml_input_tape_read_callback, @@ -1918,6 +1931,8 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) ctx, XML_CHAR_ENCODING_NONE); if (! read_buf) { ltfsmsg(LTFS_ERR, 17014E); + if (ctx->fd >= 0) + xml_release_file_lock(vol->index_cache_path_r, ctx->fd, bk, false); free(ctx->buf); free(ctx); return -LTFS_LIBXML2_FAILURE; @@ -1928,6 +1943,8 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) if (! reader) { ltfsmsg(LTFS_ERR, 17015E); xmlFreeParserInputBuffer(read_buf); + if (ctx->fd >= 0) + xml_release_file_lock(vol->index_cache_path_r, ctx->fd, bk, false); free(ctx->buf); free(ctx); return -LTFS_LIBXML2_FAILURE; @@ -1940,6 +1957,8 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) ltfsmsg(LTFS_ERR, 17015E); xmlFreeTextReader(reader); xmlFreeParserInputBuffer(read_buf); + if (ctx->fd >= 0) + xml_release_file_lock(vol->index_cache_path_r, ctx->fd, bk, false); free(ctx->buf); free(ctx); return -LTFS_LIBXML2_FAILURE; @@ -1951,7 +1970,7 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) doc = xmlTextReaderCurrentDoc(reader); /* Generate the Index. */ - ret = _xml_parse_schema(reader, vol->index, vol); + ret = _xml_parse_schema(reader, skip_dir, vol->index, vol); if (ctx->err_code < 0) { /* Error happens while reading tape */ ltfsmsg(LTFS_ERR, 17273E, ctx->err_code); @@ -1976,6 +1995,9 @@ int xml_schema_from_tape(uint64_t eod_pos, struct ltfs_volume *vol) xmlFreeTextReader(reader); xmlFreeParserInputBuffer(read_buf); + if (ctx->fd >= 0) + xml_release_file_lock(vol->index_cache_path_r, ctx->fd, bk, false); + free(ctx->buf); free(ctx); diff --git a/src/libltfs/xml_writer.c b/src/libltfs/xml_writer.c index a0554dab..53ae0de0 100644 --- a/src/libltfs/xml_writer.c +++ b/src/libltfs/xml_writer.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -48,6 +48,10 @@ ** IBM Almaden Research Center ** lucasvr@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -61,11 +65,6 @@ #include "pathname.h" #include "arch/time_internal.h" -/* O_BINARY is defined only in MinGW */ -#ifndef O_BINARY -#define O_BINARY 0 -#endif - /** * Format a raw timespec structure for the XML file. */ @@ -321,7 +320,7 @@ int xml_acquire_file_lock(const char *file, int *fd, int *bk_fd, bool is_write) /* Open specified file to lock */ *fd = open(file, O_RDWR | O_CREAT | O_BINARY, - S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH ); + S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP ); if (*fd < 0) { /* Failed to open the advisory lock '%s' (%d) */ errno_save = errno; @@ -397,7 +396,7 @@ int xml_acquire_file_lock(const char *file, int *fd, int *bk_fd, bool is_write) } *bk_fd = open(backup_file, O_RDWR | O_CREAT | O_BINARY | O_TRUNC, - S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH ); + S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP ); if (*bk_fd < 0) { ltfsmsg(LTFS_ERR, 17246E, "backup file creation", errno); errno_save = errno; diff --git a/src/libltfs/xml_writer_libltfs.c b/src/libltfs/xml_writer_libltfs.c index 8dffcc4b..5f8f76b1 100644 --- a/src/libltfs/xml_writer_libltfs.c +++ b/src/libltfs/xml_writer_libltfs.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -62,11 +62,6 @@ #include "pathname.h" #include "arch/time_internal.h" -/* O_BINARY is defined only in MinGW */ -#ifndef O_BINARY -#define O_BINARY 0 -#endif - /* Structure to control EE's file offset cache and sync file list */ struct ltfsee_cache { @@ -400,24 +395,24 @@ static int _xml_write_dirtree(xmlTextWriterPtr writer, struct dentry *dir, HASH_ITER(hh, dir->child_list, list_ptr, list_tmp) { if (list_ptr->d->isdir) { - if (list_ptr->d->vol->index_cache_path && !strcmp(list_ptr->d->name.name, ".LTFSEE_DATA")) { - ret = asprintf(&offset_name, "%s.%s", list_ptr->d->vol->index_cache_path, "offsetcache.new"); + if (list_ptr->d->vol->index_cache_path_w && !strcmp(list_ptr->d->name.name, ".LTFSEE_DATA")) { + ret = asprintf(&offset_name, "%s.%s", list_ptr->d->vol->index_cache_path_w, "offsetcache.new"); if (ret > 0) { offset->fp = fopen(offset_name, "w"); free(offset_name); if (!offset->fp) - ltfsmsg(LTFS_WARN, 17248W, "offset cache", list_ptr->d->vol->index_cache_path); + ltfsmsg(LTFS_WARN, 17248W, "offset cache", list_ptr->d->vol->index_cache_path_w); } else - ltfsmsg(LTFS_WARN, 17247W, "offset cache", list_ptr->d->vol->index_cache_path); + ltfsmsg(LTFS_WARN, 17247W, "offset cache", list_ptr->d->vol->index_cache_path_w); - ret = asprintf(&sync_name, "%s.%s", list_ptr->d->vol->index_cache_path, "synclist.new"); + ret = asprintf(&sync_name, "%s.%s", list_ptr->d->vol->index_cache_path_w, "synclist.new"); if (ret > 0) { sync->fp = fopen(sync_name, "w"); free(sync_name); if (!sync->fp) - ltfsmsg(LTFS_WARN, 17248W, "sync list", list_ptr->d->vol->index_cache_path); + ltfsmsg(LTFS_WARN, 17248W, "sync list", list_ptr->d->vol->index_cache_path_w); } else - ltfsmsg(LTFS_WARN, 17247W, "sync list", list_ptr->d->vol->index_cache_path); + ltfsmsg(LTFS_WARN, 17247W, "sync list", list_ptr->d->vol->index_cache_path_w); } xml_mktag(_xml_write_dirtree(writer, list_ptr->d, idx, offset, sync), -1); @@ -726,7 +721,7 @@ static int _commit_offset_caches(const char* path, const struct ltfs_index *idx) unlink(offset_name); rename(offset_new, offset_name); fd = open(offset_name, O_RDWR | O_BINARY, - S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH); + S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP ); if (fd >= 0) { fsync(fd); close(fd); @@ -747,7 +742,7 @@ static int _commit_offset_caches(const char* path, const struct ltfs_index *idx) unlink(sync_name); rename(sync_new, sync_name); fd = open(sync_name, O_RDWR | O_BINARY, - S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP | S_IROTH | S_IWOTH); + S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP ); if (fd >= 0) { fsync(fd); close(fd); @@ -844,8 +839,8 @@ int xml_schema_to_tape(char *reason, struct ltfs_volume *vol) out_ctx->fd = -1; out_ctx->errno_fd = 0; - if (vol->index_cache_path) - xml_acquire_file_lock(vol->index_cache_path, &out_ctx->fd, &bk, true); + if (vol->index_cache_path_w) + xml_acquire_file_lock(vol->index_cache_path_w, &out_ctx->fd, &bk, true); out_ctx->buf_size = vol->label->blocksize; out_ctx->buf_used = 0; @@ -859,7 +854,7 @@ int xml_schema_to_tape(char *reason, struct ltfs_volume *vol) if (! write_buf) { ltfsmsg(LTFS_ERR, 17053E); if (out_ctx->fd >= 0) - xml_release_file_lock(vol->index_cache_path, out_ctx->fd, bk, false); + xml_release_file_lock(vol->index_cache_path_w, out_ctx->fd, bk, false); free(out_ctx->buf); free(out_ctx); return -LTFS_LIBXML2_FAILURE; @@ -870,7 +865,7 @@ int xml_schema_to_tape(char *reason, struct ltfs_volume *vol) if (! writer) { ltfsmsg(LTFS_ERR, 17054E); if (out_ctx->fd >= 0) - xml_release_file_lock(vol->index_cache_path, out_ctx->fd, bk, false); + xml_release_file_lock(vol->index_cache_path_w, out_ctx->fd, bk, false); xmlOutputBufferClose(write_buf); free(out_ctx->buf); free(out_ctx); @@ -893,7 +888,7 @@ int xml_schema_to_tape(char *reason, struct ltfs_volume *vol) else if (out_ctx->errno_fd) ret = out_ctx->errno_fd; if (out_ctx->fd >= 0) - xml_release_file_lock(vol->index_cache_path, out_ctx->fd, bk, true); + xml_release_file_lock(vol->index_cache_path_w, out_ctx->fd, bk, true); } else { /* New index is successfully sent to the internal buffer of tape drive */ immed = (strcmp(reason, SYNC_FORMAT) == 0); /* Use immediate write FM only at format */ @@ -903,14 +898,14 @@ int xml_schema_to_tape(char *reason, struct ltfs_volume *vol) * All buffered data, new index and following FM is written on tape correctly. * It's time to unveil the offset cache and sync cache to other programs. */ - if (vol->index_cache_path) - _commit_offset_caches(vol->index_cache_path, vol->index); + if (vol->index_cache_path_w) + _commit_offset_caches(vol->index_cache_path_w, vol->index); } else { ltfsmsg(LTFS_ERR, 11084E, ret); } if (out_ctx->fd >= 0) - xml_release_file_lock(vol->index_cache_path, out_ctx->fd, bk, false); + xml_release_file_lock(vol->index_cache_path_w, out_ctx->fd, bk, false); } /* Update the creator string */ @@ -928,7 +923,7 @@ int xml_schema_to_tape(char *reason, struct ltfs_volume *vol) } else { ltfsmsg(LTFS_ERR, 10001E, "xml_schema_to_tape: creator string"); xmlFreeTextWriter(writer); - xml_release_file_lock(vol->index_cache_path, out_ctx->fd, bk, true); + xml_release_file_lock(vol->index_cache_path_w, out_ctx->fd, bk, true); ret = -LTFS_NO_MEMORY; } diff --git a/src/ltfs_copyright.h b/src/ltfs_copyright.h index ff128c89..97769fb2 100644 --- a/src/ltfs_copyright.h +++ b/src/ltfs_copyright.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -44,6 +44,10 @@ ** IBM Almaden Research Center ** bbiskebo@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -54,12 +58,16 @@ extern "C" { #endif -#define LTFS_COPYRIGHT_0 "Licensed Materials - Property of IBM" -#define LTFS_COPYRIGHT_1 "IBM Spectrum Archive Single Drive Edition" -#define LTFS_COPYRIGHT_2 "(C) Copyright IBM Corp. 2011, 2017 All Rights Reserved" -#define LTFS_COPYRIGHT_3 "US Government Users Restricted Rights - Use, duplication or " -#define LTFS_COPYRIGHT_4 "disclosure restricted by GSA ADP Schedule Contract with" -#define LTFS_COPYRIGHT_5 "IBM Corp." +#define LTFS_COPYRIGHT_0 "Copyright 2010, 2022 The LTFS project. All rights reserved." +#define LTFS_COPYRIGHT_1 "Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:" +#define LTFS_COPYRIGHT_2 "* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer." +#define LTFS_COPYRIGHT_3 "* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution." +#define LTFS_COPYRIGHT_4 "* Neither the name of the nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. " +#define LTFS_COPYRIGHT_5 "THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED " \ + "WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES " \ + "(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND " \ + "ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS " \ + "SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE." #ifdef __cplusplus } diff --git a/src/ltfs_fuse.c b/src/ltfs_fuse.c index db4d3a5c..00254728 100644 --- a/src/ltfs_fuse.c +++ b/src/ltfs_fuse.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -1166,9 +1166,6 @@ void ltfs_fuse_umount(void *userdata) ltfs_unmount(SYNC_UNMOUNT, priv->data); - if (priv->capture_index) - ltfs_save_index_to_disk(priv->work_directory, SYNC_UNMOUNT, false, priv->data); - ltfs_request_trace(FUSE_REQ_EXIT(REQ_UNMOUNT), 0, 0); } diff --git a/src/ltfs_fuse.h b/src/ltfs_fuse.h index 07a6cd35..d6e97b2c 100644 --- a/src/ltfs_fuse.h +++ b/src/ltfs_fuse.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -48,6 +48,10 @@ ** IBM Almaden Research Center ** lucasvr@us.ibm.com ** +** Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** ************************************************************************************* */ @@ -127,7 +131,7 @@ struct ltfs_fuse_data { unsigned int rollback_gen; /**< Target generation to roll back mount */ int release_device; /**< Release device? */ int allow_other; /**< Allow all users to access the volume? */ - int capture_index; /**< Capture index information to work directory at unmount */ + char *capture_dir; /**< Directory to capture index information */ char *symlink_str; /**< Symbolic Link type fetched by option (live or posix)*/ char *str_append_only_mode; /**< option sting of scsi_append_only_mode */ int append_only_mode; /**< Use append-only mode */ diff --git a/src/main.c b/src/main.c index 6b8298be..86909b0a 100644 --- a/src/main.c +++ b/src/main.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -133,7 +133,7 @@ static struct fuse_opt ltfs_options[] = { LTFS_OPT("release_device", release_device, 1), LTFS_OPT("allow_other", allow_other, 1), LTFS_OPT("noallow_other", allow_other, 0), - LTFS_OPT("capture_index", capture_index, 1), + LTFS_OPT("capture_index=%s", capture_dir, 0), LTFS_OPT("symlink_type=%s", symlink_str, 0), LTFS_OPT("scsi_append_only_mode=%s", str_append_only_mode, 0), LTFS_OPT_KEY("-a", KEY_ADVANCED_HELP), @@ -146,35 +146,39 @@ static struct fuse_opt ltfs_options[] = { void single_drive_advanced_usage(const char *default_driver, struct ltfs_fuse_data *priv) { - ltfsresult(14401I); /* LTFS options: */ - ltfsresult(14413I, LTFS_CONFIG_FILE); /* -o config_file= */ - ltfsresult(14404I, LTFS_DEFAULT_WORK_DIR); /* -o work_directory= */ - ltfsresult(14414I); /* -o atime */ - ltfsresult(14440I); /* -o noatime */ - ltfsresult(14415I, default_driver); /* -o tape_backend= */ + ltfsresult(14401I); /* LTFS options: */ + ltfsresult(14403I); /* -o devname= */ + ltfsresult(14413I, LTFS_CONFIG_FILE); /* -o config_file= */ + ltfsresult(14404I, LTFS_DEFAULT_WORK_DIR); /* -o work_directory= */ + ltfsresult(14414I); /* -o atime */ + ltfsresult(14440I); /* -o noatime */ + ltfsresult(14415I, default_driver); /* -o tape_backend= */ ltfsresult(14416I, config_file_get_default_plugin("iosched", priv->config)); /* -o iosched_backend= */ - ltfsresult(14455I, config_file_get_default_plugin("kmi", priv->config)); /* -o kmi_backend= */ - ltfsresult(14417I); /* -o umask= */ - ltfsresult(14418I); /* -o fmask= */ - ltfsresult(14419I); /* -o dmask= */ + ltfsresult(14455I, config_file_get_default_plugin("kmi", priv->config)); /* -o kmi_backend= */ + ltfsresult(14417I); /* -o umask= */ + ltfsresult(14418I); /* -o fmask= */ + ltfsresult(14419I); /* -o dmask= */ ltfsresult(14420I, LTFS_MIN_CACHE_SIZE_DEFAULT); /* -o min_pool_size= */ ltfsresult(14421I, LTFS_MAX_CACHE_SIZE_DEFAULT); /* -o max_pool_size= */ - ltfsresult(14422I); /* -o rules= */ - ltfsresult(14423I); /* -o quiet */ - ltfsresult(14405I); /* -o trace */ - ltfsresult(14467I); /* -o syslogtrace */ - ltfsresult(14424I); /* -o fulltrace */ - ltfsresult(14441I, LTFS_INFO); /* -o verbose= */ - ltfsresult(14425I); /* -o eject */ - ltfsresult(14439I); /* -o noeject */ - ltfsresult(14427I, LONG_MAX / 60); /* -o sync_type=type */ - ltfsresult(14443I); /* -o force_mount_no_eod */ - ltfsresult(14436I); /* -o device_list */ - ltfsresult(14437I); /* -o rollback_mount */ - ltfsresult(14448I); /* -o release_device */ - ltfsresult(14456I); /* -o capture_index */ - ltfsresult(14463I); /* -o scsi_append_only_mode= */ - ltfsresult(14406I); /* -a */ + ltfsresult(14422I); /* -o rules= */ + ltfsresult(14423I); /* -o quiet */ + ltfsresult(14405I); /* -o trace */ + ltfsresult(14467I); /* -o syslogtrace */ + ltfsresult(14424I); /* -o fulltrace */ + ltfsresult(14441I, LTFS_INFO); /* -o verbose= */ + ltfsresult(14425I); /* -o eject */ + ltfsresult(14439I); /* -o noeject */ + ltfsresult(14427I, LONG_MAX / 60); /* -o sync_type=type */ + ltfsresult(14443I); /* -o force_mount_no_eod */ + ltfsresult(14436I); /* -o device_list */ + ltfsresult(14437I); /* -o rollback_mount */ + ltfsresult(14448I); /* -o release_device */ + ltfsresult(14461I); /* -o symlink_type=type */ + ltfsresult(14456I); /* -o capture_index */ + ltfsresult(14463I); /* -o scsi_append_only_mode= */ + ltfsresult(14406I); /* -a */ + ltfsresult(14407I); /* -V, --version */ + ltfsresult(14408I); /* -h, --help */ /* TODO: future use for WORM */ /* set worm rollback flag and rollback_str by this option */ /* ltfsresult(14468I); */ /* -o rollback_mount_no_eod */ @@ -194,30 +198,30 @@ void usage(char *progname, struct ltfs_fuse_data *priv) if (ret == 0) default_device = ltfs_default_device_name(priv->tape_plugin.ops); - ltfsresult(14400I, progname); /* usage: %s mountpoint [options] */ + ltfsresult(14400I, progname); /* usage: %s mountpoint [options] */ fprintf(stderr, "\n"); - ltfsresult(14401I); /* LTFS options: */ + ltfsresult(14401I); /* LTFS options: */ if (default_device) - ltfsresult(14402I, default_device); /* -o devname= */ + ltfsresult(14402I, default_device); /* -o devname= */ else - ltfsresult(14403I); /* -o devname= */ - ltfsresult(14404I, LTFS_DEFAULT_WORK_DIR); /* -o work_directory= */ - ltfsresult(14405I); /* -o trace */ - ltfsresult(14425I); /* -o eject */ - ltfsresult(14427I, LONG_MAX / 60); /* -o sync_type=type */ - ltfsresult(14443I); /* -o force_mount_no_eod */ - ltfsresult(14436I); /* -o device_list */ - ltfsresult(14437I); /* -o rollback_mount */ - ltfsresult(14448I); /* -o release_device */ - ltfsresult(14461I); /* -o symlink_type=type */ - ltfsresult(14406I); /* -a */ - ltfsresult(14407I); /* -V, --version */ - ltfsresult(14408I); /* -h, --help */ + ltfsresult(14403I); /* -o devname= */ + ltfsresult(14404I, LTFS_DEFAULT_WORK_DIR); /* -o work_directory= */ + ltfsresult(14405I); /* -o trace */ + ltfsresult(14425I); /* -o eject */ + ltfsresult(14427I, LONG_MAX / 60); /* -o sync_type=type */ + ltfsresult(14443I); /* -o force_mount_no_eod */ + ltfsresult(14436I); /* -o device_list */ + ltfsresult(14437I); /* -o rollback_mount */ + ltfsresult(14448I); /* -o release_device */ + ltfsresult(14461I); /* -o symlink_type=type */ + ltfsresult(14406I); /* -a */ + ltfsresult(14407I); /* -V, --version */ + ltfsresult(14408I); /* -h, --help */ fprintf(stderr, "\n"); - ltfsresult(14409I); /* FUSE options: */ - ltfsresult(14410I); /* -o umask=M */ - ltfsresult(14411I); /* -o uid=N */ - ltfsresult(14412I); /* -o gid=N */ + ltfsresult(14409I); /* FUSE options: */ + ltfsresult(14410I); /* -o umask=M */ + ltfsresult(14411I); /* -o uid=N */ + ltfsresult(14412I); /* -o gid=N */ fprintf(stderr, "\n"); fprintf(stderr, "\n"); @@ -877,11 +881,14 @@ int main(int argc, char **argv) /* Make sure we have a device name */ if (! priv->devname) { - priv->devname = ltfs_default_device_name(priv->tape_plugin.ops); - if (! priv->devname) { - /* The backend \'%s\' does not have a default device */ - ltfsmsg(LTFS_ERR, 14009E, priv->tape_backend_name); - return 1; + /* Accept no devname when accessible index file is specified by '-o rollback_mount' */ + if ( !priv->rollback_str || access(priv->rollback_str, R_OK) ) { + priv->devname = ltfs_default_device_name(priv->tape_plugin.ops); + if (! priv->devname) { + /* The backend \'%s\' does not have a default device */ + ltfsmsg(LTFS_ERR, 14009E, priv->tape_backend_name); + return 1; + } } } @@ -922,9 +929,10 @@ int main(int argc, char **argv) int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) { - int ret, altret; + int ret, altret, fd = -1; char *index_rules_utf8; - char fsname[strlen(priv->devname) + 16]; + char *fsname_base = "-ofsname=ltfs:"; + char *fsname; char *invalid_start; #ifdef __APPLE__ char *opt_volname = NULL; @@ -934,6 +942,18 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) int i; bool is_worm = false, is_ro = false; + if (priv->devname) { + fsname = calloc(1, strlen(fsname_base) + strlen(priv->devname) + 1); + } else if (priv->rollback_str) { + fsname = calloc(1, strlen(fsname_base) + strlen(priv->rollback_str) + 1); + } else { + fsname = calloc(1, strlen(fsname_base) + 1); + } + if (!fsname) { + /* Memory allocation failed */ + ltfsmsg(LTFS_ERR, 10001E, "fsname"); + return -ENOMEM; + } /* Setup signal handler to terminate cleanly */ ret = ltfs_set_signal_handlers(); if (ret < 0) { @@ -943,11 +963,13 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) /* Validate rollback_mount option */ if (priv->rollback_str) { - errno = 0; - priv->rollback_gen = strtoul(priv->rollback_str, &invalid_start, 0); - if( (*invalid_start != '\0') || priv->rollback_gen == 0 ) { - ltfsmsg(LTFS_ERR, 14091E, priv->rollback_str); - return 1; + if (access(priv->rollback_str, R_OK)) { + errno = 0; + priv->rollback_gen = strtoul(priv->rollback_str, &invalid_start, 0); + if( (*invalid_start != '\0') || priv->rollback_gen == 0 ) { + ltfsmsg(LTFS_ERR, 14091E, priv->rollback_str); + return 1; + } } } @@ -983,7 +1005,12 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) } /* Set file system name to "ltfs:devname" in case FUSE doesn't pick it up */ - snprintf(fsname, sizeof(fsname), "-ofsname=ltfs:%s", priv->devname); + strcpy(fsname, fsname_base); + if (priv->devname) { + strcat(fsname, priv->devname); + } else if (priv->rollback_str) { + strcat(fsname, priv->rollback_str); + } ret = fuse_opt_add_arg(args, fsname); if (ret < 0) { /* Could not enable FUSE option */ @@ -1000,145 +1027,163 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) ltfs_use_atime(priv->atime, priv->data); ltfs_set_work_dir(priv->work_directory, priv->data); - if (ltfs_device_open(priv->devname, priv->tape_plugin.ops, priv->data) < 0) { - /* Could not open device */ - ltfsmsg(LTFS_ERR, 10004E, priv->devname); - ltfs_volume_free(&priv->data); - return 1; - } - - if (priv->release_device) { - ltfs_release_medium(priv->data); - ltfs_device_close(priv->data); - ltfs_volume_free(&priv->data); - return 0; - } - - /* Parse backend options */ - if (ltfs_parse_tape_backend_opts(args, priv->data)) { - /* Backend option parsing failed */ - ltfsmsg(LTFS_ERR, 14012E); - ltfs_volume_free(&priv->data); - return 1; - } - - if (priv->kmi_backend_name) { - if (kmi_init(&priv->kmi_plugin, priv->data) < 0) { - /* Encryption function disabled. */ - ltfsmsg(LTFS_ERR, 14089E); + if (priv->devname) { + if (ltfs_device_open(priv->devname, priv->tape_plugin.ops, priv->data) < 0) { + /* Could not open device */ + ltfsmsg(LTFS_ERR, 10004E, priv->devname); ltfs_volume_free(&priv->data); return 1; } - if (ltfs_parse_kmi_backend_opts(args, priv->data)) { + if (priv->release_device) { + ltfs_release_medium(priv->data); + ltfs_device_close(priv->data); + ltfs_volume_free(&priv->data); + return 0; + } + + /* Parse backend options */ + if (ltfs_parse_tape_backend_opts(args, priv->data)) { /* Backend option parsing failed */ - ltfsmsg(LTFS_ERR, 14090E); + ltfsmsg(LTFS_ERR, 14012E); ltfs_volume_free(&priv->data); return 1; } - if (tape_clear_key(priv->data->device, priv->data->kmi_handle) < 0) - return 1; - } + if (priv->kmi_backend_name) { + if (kmi_init(&priv->kmi_plugin, priv->data) < 0) { + /* Encryption function disabled. */ + ltfsmsg(LTFS_ERR, 14089E); + ltfs_volume_free(&priv->data); + return 1; + } - /* Setup tape drive */ - ltfs_load_tape(priv->data); - ret = ltfs_wait_device_ready(priv->data); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 14075E); - ltfs_volume_free(&priv->data); - return 1; - } + if (ltfs_parse_kmi_backend_opts(args, priv->data)) { + /* Backend option parsing failed */ + ltfsmsg(LTFS_ERR, 14090E); + ltfs_volume_free(&priv->data); + return 1; + } - priv->data->append_only_mode = (bool)priv->append_only_mode; - if (ltfs_setup_device(priv->data)) { - ltfsmsg(LTFS_ERR, 14075E); - ltfs_volume_free(&priv->data); - return 1; - } + if (tape_clear_key(priv->data->device, priv->data->kmi_handle) < 0) + return 1; + } - /* Check EOD validation is skipped or not */ - if (priv->skip_eod_check) { - ltfsmsg(LTFS_INFO, 14076I); - ltfsmsg(LTFS_INFO, 14077I); - ltfs_set_eod_check(! priv->skip_eod_check, priv->data); - } + /* Setup tape drive */ + ltfs_load_tape(priv->data); + ret = ltfs_wait_device_ready(priv->data); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 14075E); + ltfs_volume_free(&priv->data); + return 1; + } - /* Validate symbolic link type */ - priv->data->livelink = false; - if (priv->symlink_str) { - if (strcasecmp(priv->symlink_str, "live") == 0) - priv->data->livelink = true; - else if (strcasecmp(priv->symlink_str, "posix") == 0) - priv->data->livelink = false; - else { - ltfsmsg(LTFS_ERR, 14093E, priv->symlink_str); + priv->data->append_only_mode = (bool)priv->append_only_mode; + if (ltfs_setup_device(priv->data)) { + ltfsmsg(LTFS_ERR, 14075E); + ltfs_volume_free(&priv->data); return 1; } - ltfsmsg(LTFS_INFO, 14092I, priv->symlink_str); - } - /* Mount the volume */ - ltfs_set_traverse_mode(TRAVERSE_BACKWARD, priv->data); - if (ltfs_mount(false, false, false, false, priv->rollback_gen, priv->data) < 0) { - ltfsmsg(LTFS_ERR, 14013E); - ltfs_volume_free(&priv->data); - return 1; - } + /* Check EOD validation is skipped or not */ + if (priv->skip_eod_check) { + ltfsmsg(LTFS_INFO, 14076I); + ltfsmsg(LTFS_INFO, 14077I); + ltfs_set_eod_check(! priv->skip_eod_check, priv->data); + } - ret = tape_get_worm_status(priv->data->device, &is_worm); - if (ret != 0 || is_worm) { - ltfsmsg(LTFS_ERR, 14116E, ret); - ltfs_volume_free(&priv->data); - return 1; - } + /* Validate symbolic link type */ + priv->data->livelink = false; + if (priv->symlink_str) { + if (strcasecmp(priv->symlink_str, "live") == 0) + priv->data->livelink = true; + else if (strcasecmp(priv->symlink_str, "posix") == 0) + priv->data->livelink = false; + else { + ltfsmsg(LTFS_ERR, 14093E, priv->symlink_str); + return 1; + } + ltfsmsg(LTFS_INFO, 14092I, priv->symlink_str); + } - /* Set up index criteria */ - if (priv->index_rules) { - ret = pathname_format(priv->index_rules, &index_rules_utf8, false, false); - if (ret < 0) { - /* Could not format data placement rules. */ - ltfsmsg(LTFS_ERR, 14016E, ret); - ltfs_volume_free(&priv->data); - return 1; + /* Mount the volume */ + ltfs_set_traverse_mode(TRAVERSE_BACKWARD, priv->data); + if (priv->rollback_str) { + if (ltfs_mount_indexfile(priv->rollback_str, true, priv->data) < 0) { + ltfsmsg(LTFS_ERR, 14013E, "index file"); + ltfs_volume_free(&priv->data); + return 1; + } + } else { + if (ltfs_mount(false, false, false, false, priv->rollback_gen, priv->data) < 0) { + ltfsmsg(LTFS_ERR, 14013E, "device"); + ltfs_volume_free(&priv->data); + return 1; + } } - ret = ltfs_override_policy(index_rules_utf8, false, priv->data); - free(index_rules_utf8); - if (ret == -LTFS_POLICY_IMMUTABLE) { - /* Volume doesn't allow override. Ignoring user-specified criteria. */ - ltfsmsg(LTFS_WARN, 14015W); - } else if (ret < 0) { - /* Could not parse data placement rules */ - ltfsmsg(LTFS_ERR, 14017E, ret); + + ret = tape_get_worm_status(priv->data->device, &is_worm); + if (ret != 0 || is_worm) { + ltfsmsg(LTFS_ERR, 14116E, ret); ltfs_volume_free(&priv->data); return 1; } - } - /* Configure I/O scheduler cache */ - ltfs_set_scheduler_cache(priv->min_pool_size, priv->max_pool_size, priv->data); - - /* mount read-only if underlying medium is write-protected */ - ret = ltfs_get_tape_readonly(priv->data); - switch (ret) { - case 0: - case -LTFS_WRITE_PROTECT: - case -LTFS_WRITE_ERROR: - case -LTFS_NO_SPACE: - case -LTFS_LESS_SPACE: - case -LTFS_RDONLY_DEN_DRV: - /* Do nothing */ - break; - default: - /* Fail immidiatly when return code is NOT success or NOT possible R/O related errors */ - /* Could not get read-only status of medium */ - ltfsmsg(LTFS_ERR, 14018E); + /* Set up index criteria */ + if (priv->index_rules) { + ret = pathname_format(priv->index_rules, &index_rules_utf8, false, false); + if (ret < 0) { + /* Could not format data placement rules. */ + ltfsmsg(LTFS_ERR, 14016E, ret); + ltfs_volume_free(&priv->data); + return 1; + } + ret = ltfs_override_policy(index_rules_utf8, false, priv->data); + free(index_rules_utf8); + if (ret == -LTFS_POLICY_IMMUTABLE) { + /* Volume doesn't allow override. Ignoring user-specified criteria. */ + ltfsmsg(LTFS_WARN, 14015W); + } else if (ret < 0) { + /* Could not parse data placement rules */ + ltfsmsg(LTFS_ERR, 14017E, ret); + ltfs_volume_free(&priv->data); + return 1; + } + } + + /* Configure I/O scheduler cache */ + ltfs_set_scheduler_cache(priv->min_pool_size, priv->max_pool_size, priv->data); + + /* mount read-only if underlying medium is write-protected */ + ret = ltfs_get_tape_readonly(priv->data); + switch (ret) { + case 0: + case -LTFS_WRITE_PROTECT: + case -LTFS_WRITE_ERROR: + case -LTFS_NO_SPACE: + case -LTFS_LESS_SPACE: + case -LTFS_RDONLY_DEN_DRV: + /* Do nothing */ + break; + default: + /* Fail immidiatly when return code is NOT success or NOT possible R/O related errors */ + /* Could not get read-only status of medium */ + ltfsmsg(LTFS_ERR, 14018E); + ltfs_volume_free(&priv->data); + return 1; + break; + } + } else { + /* try to mount from index file (meta-only mount) */ + if (ltfs_mount_indexfile(priv->rollback_str, false, priv->data) < 0) { + ltfsmsg(LTFS_ERR, 14013E, "index file"); ltfs_volume_free(&priv->data); return 1; - break; + } + ret = 0; } - if (ret < 0 || priv->rollback_gen != 0) { + if (ret < 0 || priv->rollback_gen || priv->rollback_str) { switch (ret) { case -LTFS_WRITE_PROTECT: case -LTFS_WRITE_ERROR: @@ -1171,9 +1216,25 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) is_ro = true; break; default: - /* Rollback mount is specified */ - ltfsmsg(LTFS_INFO, 14072I, priv->rollback_gen); - is_ro = true; + if (!ret && priv->rollback_gen) { + /* Rollback mount is specified */ + ltfsmsg(LTFS_INFO, 14072I, priv->rollback_gen); + is_ro = true; + } else if (!ret && priv->rollback_str) { + if (priv->devname) { + /* Rollback mount (index mount) is specified */ + ltfsmsg(LTFS_INFO, 14119I, priv->rollback_str); + } else { + /* Rollback mount (meta-only mount) is specified */ + ltfsmsg(LTFS_INFO, 14117I, priv->rollback_str); + } + is_ro = true; + } else { + /* Unexpected condition */ + ltfsmsg(LTFS_ERR, 14118E, ret); + ltfs_volume_free(&priv->data); + return 1; + } break; } @@ -1188,10 +1249,42 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) } } + /* Configure index capturing */ + if (priv->capture_dir) { + if (HAVE_BARCODE(priv->data)) + ret = asprintf(&priv->data->index_cache_path_w, "%s/%s.schema", + priv->capture_dir, priv->data->label->barcode); + else + ret = asprintf(&priv->data->index_cache_path_w, "%s/%s.schema", + priv->capture_dir, priv->data->label->vol_uuid); + + if (ret > 0) { + fd = open(priv->data->index_cache_path_w, O_WRONLY | O_BINARY | O_CREAT, + S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP); + if (fd < 0) { + ltfsmsg(LTFS_WARN, 14120W, priv->capture_dir, errno); + free(priv->data->index_cache_path_w); + priv->data->index_cache_path_w = NULL; + } else { + ltfsmsg(LTFS_INFO, 14121I, priv->data->index_cache_path_w); + close(fd); + fd = -1; + } + } else { + ltfsmsg(LTFS_ERR, 10001E, "capture_dir"); + ltfs_volume_free(&priv->data); + return 1; + } + } else { + ltfsmsg(LTFS_INFO, 14122I); + ret = 0; + } + /* Cleanup signal handler */ ret = ltfs_unset_signal_handlers(); if (ret < 0) { ltfsmsg(LTFS_ERR, 10014E); + ltfs_volume_free(&priv->data); return 1; } @@ -1250,7 +1343,7 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) return 1; } - if (priv->eject) + if (priv->devname && priv->eject) ltfs_eject_tape(false, priv->data); /* close the volume */ @@ -1258,7 +1351,8 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) if (opt_volname) free(opt_volname); #endif - ltfs_device_close(priv->data); + if (priv->devname) + ltfs_device_close(priv->data); ltfs_volume_free(&priv->data); ltfs_unset_signal_handlers(); diff --git a/src/utils/ltfsck.c b/src/utils/ltfsck.c index d8e2b296..31214a1a 100644 --- a/src/utils/ltfsck.c +++ b/src/utils/ltfsck.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2022 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -96,27 +96,27 @@ enum { }; struct other_check_opts { - struct config_file *config; /**< Configurate data read from the global LTFS config file. */ - char *devname; /**< Device to format */ - char *backend_path; /**< Path to tape backend shared library */ - char *kmi_backend_name; /**< Name or path to the key manager interface backend library */ - int op_mode; /**< Operation mode */ - int search_mode; /**< Search mode for index */ - char *str_gen; /**< Rollback point specified by command line (generation)*/ - unsigned int point_gen; /**< Rollback point (generation) */ - bool erase_history; /**< overwrite existing data at rollback */ - bool recover_blocks; /**< Recover unreferenced blocks at the ends of the partitions? */ - bool deep_recovery; /**< Recover EOD missing cartridge? */ - int verbosity; /**< Print extra messages? */ - char *prg_name; /**< Program name */ - bool quiet; /**< Suppress information messages */ - bool trace; /**< Generate debug output */ - bool syslogtrace; /**< Generate debug output to stderr and syslog*/ - bool fulltrace; /**< Trace function calls */ - int traverse_mode; /**< Traverse strategy for listing index */ - bool full_index_info; /**< Print full index infomation in list mode */ - bool capture_index; /**< Capture index in list mode */ - bool salvage_points; /**< List rollback points from no-EOD cartridge? */ + struct config_file *config; /**< Configurate data read from the global LTFS config file. */ + char *devname; /**< Device to format */ + char *backend_path; /**< Path to tape backend shared library */ + char *kmi_backend_name; /**< Name or path to the key manager interface backend library */ + int op_mode; /**< Operation mode */ + int search_mode; /**< Search mode for index */ + char *str_gen; /**< Rollback point specified by command line (generation)*/ + unsigned int point_gen; /**< Rollback point (generation) */ + bool erase_history; /**< overwrite existing data at rollback */ + bool recover_blocks; /**< Recover unreferenced blocks at the ends of the partitions? */ + bool deep_recovery; /**< Recover EOD missing cartridge? */ + int verbosity; /**< Print extra messages? */ + char *prg_name; /**< Program name */ + bool quiet; /**< Suppress information messages */ + bool trace; /**< Generate debug output */ + bool syslogtrace; /**< Generate debug output to stderr and syslog*/ + bool fulltrace; /**< Trace function calls */ + int traverse_mode; /**< Traverse strategy for listing index */ + bool full_index_info; /**< Print full index infomation in list mode */ + char *capture_dir; /**< Capture index in list mode and it's directory */ + bool salvage_points; /**< List rollback points from no-EOD cartridge? */ }; struct index_info @@ -167,7 +167,7 @@ static struct option long_options[] = { {"generation", 1, 0, 'g'}, {"traverse", 1, 0, 'v'}, {"kmi-backend", 1, 0, '-'}, - {"capture-index", 0, 0, '+'}, + {"capture-index", 1, 0, '+'}, {"rollback" , 0, 0, 'r'}, {"no-rollback" , 0, 0, 'n'}, {"full-recovery" , 0, 0, 'f'}, @@ -350,7 +350,7 @@ int main(int argc, char **argv) break; case '+': opt.op_mode = MODE_LIST_POINT; - opt.capture_index = true; + opt.capture_dir = strdup(optarg); break; case 'r': opt.op_mode = MODE_ROLLBACK; @@ -840,7 +840,8 @@ void _print_index_header(bool full_info) void _print_index(struct ltfs_volume *vol, struct index_info *list, struct other_check_opts *opt) { struct tm *t_st; - int i; + char *new_path = NULL; + int i, ret; if(!opt) return; @@ -935,8 +936,26 @@ void _print_index(struct ltfs_volume *vol, struct index_info *list, struct other else printf(" No commit message\n"); - if (opt->capture_index) - ltfs_save_index_to_disk(".", NULL, true, vol); + /* Rename reading xml to official name */ + if (vol->index_cache_path_r && opt->capture_dir) { + if (HAVE_BARCODE(vol)) + ret = asprintf(&new_path, "%s/%s-%d-%c.schema", opt->capture_dir, vol->label->barcode, + vol->index->generation, list->selfptr.partition); + else + ret = asprintf(&new_path, "%s/%s-%d-%c.schema", opt->capture_dir, vol->label->vol_uuid, + vol->index->generation, list->selfptr.partition); + + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10001E, "_print_index: path"); + return; + } + + ret = rename(vol->index_cache_path_r, new_path); + if (ret < 0) { + ltfsmsg(LTFS_WARN, 16112W, vol->index_cache_path_r, new_path, errno); + } + free(new_path); + } return; } @@ -968,7 +987,7 @@ void print_index_array(struct ltfs_volume *vol, struct index_info *list, void *o while (cur) { _print_index(vol, cur, opt); - cur = cur-> next; + cur = cur->next; } return; @@ -1280,11 +1299,11 @@ int rollback(struct ltfs_volume *vol, struct other_check_opts *opt) } /* Find target index */ - ret = ltfs_traverse_index_backward(vol, ltfs_ip_id(vol), opt->point_gen, + ret = ltfs_traverse_index_backward(vol, ltfs_ip_id(vol), opt->point_gen, false, search_index_by_gen, (void *)(&(r.target_info)), (void *)opt); if (ret == -LTFS_NO_INDEX) { if (opt->erase_history) { - ret = ltfs_traverse_index_forward(vol, ltfs_dp_id(vol), opt->point_gen, + ret = ltfs_traverse_index_forward(vol, ltfs_dp_id(vol), opt->point_gen, false, search_index_by_gen, (void *)(&(r.target_info)), (void *)opt); if (ret == -LTFS_NO_INDEX) { ltfsmsg(LTFS_ERR, 16072E, ret); @@ -1294,7 +1313,7 @@ int rollback(struct ltfs_volume *vol, struct other_check_opts *opt) return LTFSCK_OPERATIONAL_ERROR;; } } else { - ret = ltfs_traverse_index_backward(vol, ltfs_dp_id(vol), opt->point_gen, + ret = ltfs_traverse_index_backward(vol, ltfs_dp_id(vol), opt->point_gen, false, search_index_by_gen, (void *)(&(r.target_info)), (void *)opt); if (ret != LTFSCK_NO_ERRORS) { ltfsmsg(LTFS_ERR, 16072E, ret); @@ -1352,7 +1371,7 @@ int rollback(struct ltfs_volume *vol, struct other_check_opts *opt) int list_rollback_points_normal(struct ltfs_volume *vol, struct other_check_opts *opt) { - int ret = LTFSCK_NO_ERRORS; + int ret = LTFSCK_NO_ERRORS, fd = -1; /* Load tape and read labels */ ret = load_tape(vol); @@ -1372,14 +1391,41 @@ int list_rollback_points_normal(struct ltfs_volume *vol, struct other_check_opts } } + /* Configure index capturing */ + if (opt->capture_dir) { + /* Confirm the directory and construct read cache path */ + ret = asprintf(&vol->index_cache_path_r, "%s/reading_index.xml", opt->capture_dir); + + if (ret > 0) { + fd = open(vol->index_cache_path_r, O_WRONLY | O_BINARY | O_CREAT, + S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP); + if (fd < 0) { + ltfsmsg(LTFS_WARN, 16113W, opt->capture_dir, errno); + free(vol->index_cache_path_r); + vol->index_cache_path_r = NULL; + } else { + ltfsmsg(LTFS_INFO, 16114I, opt->capture_dir); + close(fd); + fd = -1; + } + } else { + ltfsmsg(LTFS_ERR, 10001E, "capture_dir"); + ltfs_volume_free(&vol); + return 1; + } + } else { + ltfsmsg(LTFS_INFO, 16115I); + } + + /* Print header */ _print_index_header(opt->full_index_info); /* read index from the index partition */ if(opt->traverse_mode == TRAVERSE_FORWARD) - ret = ltfs_traverse_index_forward(vol, ltfs_ip_id(vol), opt->point_gen, + ret = ltfs_traverse_index_forward(vol, ltfs_ip_id(vol), opt->point_gen, true, print_a_index_noheader, NULL, (void *)opt); else - ret = ltfs_traverse_index_backward(vol, ltfs_ip_id(vol), opt->point_gen, + ret = ltfs_traverse_index_backward(vol, ltfs_ip_id(vol), opt->point_gen, true, print_a_index_noheader, NULL, (void *)opt); if (ret != LTFSCK_NO_ERRORS) { ltfsmsg(LTFS_ERR, 16075E, ret); @@ -1388,10 +1434,10 @@ int list_rollback_points_normal(struct ltfs_volume *vol, struct other_check_opts /* read index from the data partition */ if(opt->traverse_mode == TRAVERSE_FORWARD) - ret = ltfs_traverse_index_forward(vol, ltfs_dp_id(vol), opt->point_gen, + ret = ltfs_traverse_index_forward(vol, ltfs_dp_id(vol), opt->point_gen, true, print_a_index_noheader, NULL, (void *)opt); else - ret = ltfs_traverse_index_backward(vol, ltfs_dp_id(vol), opt->point_gen, + ret = ltfs_traverse_index_backward(vol, ltfs_dp_id(vol), opt->point_gen, true, print_a_index_noheader, NULL, (void *)opt); if (ret != LTFSCK_NO_ERRORS) { ltfsmsg(LTFS_ERR, 16076E, ret); @@ -1427,7 +1473,7 @@ int list_rollback_points_no_eod(struct ltfs_volume *vol, struct other_check_opts /* Read index from the data partition */ /* We don't need to read IP because index in DP is always newer (or same) than IP in case of WORM */ - ret = ltfs_traverse_index_no_eod(vol, ltfs_dp_id(vol), opt->point_gen, + ret = ltfs_traverse_index_no_eod(vol, ltfs_dp_id(vol), opt->point_gen, true, print_a_index_noheader, NULL, (void *)opt); if (ret != LTFSCK_NO_ERRORS) { ltfsmsg(LTFS_ERR, 16076E, ret); @@ -1487,7 +1533,7 @@ int _ltfsck_validate_options(struct other_check_opts *opt) else if (opt->traverse_mode == TRAVERSE_BACKWARD) ltfsmsg(LTFS_INFO, 16084I); - if (opt->capture_index && opt->search_mode == SEARCH_BY_GEN) { + if (opt->capture_dir && opt->search_mode == SEARCH_BY_GEN) { if (!opt->str_gen) { ltfsmsg(LTFS_ERR, 16004E); return LTFSCK_USAGE_SYNTAX_ERROR; From 376a25b0509bdc826e287d63b1f5b14a15f3ddab Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Thu, 3 Feb 2022 17:18:29 +0900 Subject: [PATCH 074/121] Fix check build break on CentOS8 --- .github/workflows/build-centos8.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build-centos8.yml b/.github/workflows/build-centos8.yml index 85a3c13a..f946a4f6 100644 --- a/.github/workflows/build-centos8.yml +++ b/.github/workflows/build-centos8.yml @@ -12,6 +12,6 @@ jobs: uses: actions/checkout@v1 - name: Build LTFS id: build - uses: LinearTapeFileSystem/CentOS8-Build@v1.3 + uses: LinearTapeFileSystem/CentOS8-Build@v1.4 with: destination: '/tmp/ltfs' From 8064838cd44f25b2b2b1b81506aaf4879773e645 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 15 Feb 2022 10:28:34 +0900 Subject: [PATCH 075/121] Format time enhancement on the LTO9 drive (#335) * Use faster setting of format/unformat in LTO9 * Introduce `--destructive` option to mkltfs * Update man pages --- man/ltfs.8 | 2 +- man/ltfs_ordered_copy.1 | 2 +- man/mkltfs.8 | 9 ++-- man/sgml/mkltfs.sgml | 8 ++++ messages/bin_mkltfs/root.txt | 1 + messages/libltfs/root.txt | 4 +- src/libltfs/ltfs.c | 28 +++++++++--- src/libltfs/ltfs.h | 4 +- src/libltfs/tape.c | 86 +++++++++++++++++++++++++++--------- src/libltfs/tape.h | 3 +- src/utils/mkltfs.c | 13 ++++-- 11 files changed, 119 insertions(+), 41 deletions(-) diff --git a/man/ltfs.8 b/man/ltfs.8 index 1466fe0b..3f435c25 100644 --- a/man/ltfs.8 +++ b/man/ltfs.8 @@ -4,7 +4,7 @@ ltfs \- File system based on a linear tape drive .SH SYNOPSIS .sp -\fBltfs\fR \fB\fImount_point\fB\fR [ \fB-o \fImount_option\fB\fR\fI...\fR ] [ \fB-V\fR ] [ \fB-h\fR ] [ \fB-a\fR ] +\fBltfs\fR \fB\fImount_point\fB\fR [ \fB-o \fImount_option\fB\fR\fI...\fR ] [ \fB-V\fR ] [ \fB-h\fR ] [ \fB-a\fR ] .SH "DESCRIPTION" .PP \fBltfs\fR is a file system for linear tape drive. diff --git a/man/ltfs_ordered_copy.1 b/man/ltfs_ordered_copy.1 index f5206629..21ccc447 100644 --- a/man/ltfs_ordered_copy.1 +++ b/man/ltfs_ordered_copy.1 @@ -4,7 +4,7 @@ ltfs_ordered_copy \- Copy files from source to destination with LTFS order optimization .SH SYNOPSIS .sp -\fBltfs_ordered_copy\fR [ \fB-p\fR ] [ \fB-r\fR ] [ \fB-t \fITARGET_DIRECTORY\fB\fR ] [ \fB--keep-tree \fICUTOFF_PREFIX\fB\fR ] [ \fB-a\fR ] [ \fB-v\fR ] [ \fB--verbose \fILOG_LEVEL\fB\fR ] [ \fB-q\fR ] [ \fB-h\fR ] [ \fBSOURCE \fR\fI...\fR ] [ \fBDESTINATION\fR ] +\fBltfs_ordered_copy\fR [ \fB-p\fR ] [ \fB-r\fR ] [ \fB-t \fITARGET_DIRECTORY\fB\fR ] [ \fB--keep-tree \fICUTOFF_PREFIX\fB\fR ] [ \fB-a\fR ] [ \fB-v\fR ] [ \fB--verbose \fILOG_LEVEL\fB\fR ] [ \fB-q\fR ] [ \fB-h\fR ] [ \fBSOURCE \fR\fI...\fR ] [ \fBDESTINATION\fR ] .SH "DESCRIPTION" .PP \fBltfs_ordered_copy\fR is a program to copy files from source to destination with LTFS diff --git a/man/mkltfs.8 b/man/mkltfs.8 index ed629fe2..e3b87f73 100644 --- a/man/mkltfs.8 +++ b/man/mkltfs.8 @@ -1,10 +1,10 @@ .\" auto-generated by docbook2man-spec from docbook-utils package -.TH "MKLTFS" "8" "19 August 2021" "LTFS" "LTFS Command Reference" +.TH "MKLTFS" "8" "08 February 2022" "LTFS" "LTFS Command Reference" .SH NAME mkltfs \- Format a tape in the drive to LTFS format .SH SYNOPSIS .sp -\fBmkltfs\fR \fB-d \fIname\fB\fR [ \fB-f\fR ] [ \fB-s \fIid\fB\fR ] [ \fB-n \fIname\fB\fR ] [ \fB-r \fIrules\fB\fR ] [ \fB-w\fR ] [ \fB-q\fR ] [ \fB-t\fR ] [ \fB-V\fR ] [ \fB-h\fR ] [ \fB-p\fR ] +\fBmkltfs\fR \fB-d \fIname\fB\fR [ \fB-f\fR ] [ \fB-s \fIid\fB\fR ] [ \fB-n \fIname\fB\fR ] [ \fB-r \fIrules\fB\fR ] [ \fB-w\fR ] [ \fB-q\fR ] [ \fB-t\fR ] [ \fB-V\fR ] [ \fB-h\fR ] [ \fB-p\fR ] .SH "DESCRIPTION" .PP \fBmkltfs\fR is a program to format a media for use with @@ -75,7 +75,7 @@ Full help, including advanced options /home/piste/ltfs05-sde/bin/mkltfs --device=/dev/IBMtape0 --rules="size=100K" /home/piste/ltfs05-sde/bin/mkltfs --device=/dev/IBMtape0 --rules="size=1M/name=*.jpg" /home/piste/ltfs05-sde/bin/mkltfs --device=/dev/IBMtape0 --rules="size=1M/name=*.jpg:*.png" - + .sp .fi .PP @@ -112,6 +112,9 @@ Enable full function call tracing (slow) \fB--long-wipe\fR Unformat the medium and erase any data on the tape by overwriting special data pattern. This operation takes over 3 hours. Once you start, you cannot interrupt it. +.TP +\fB--destructive\fR +Use destructive format/unformat. This operation takes longer time in the LTO9 drive or later because of the media optimization procedure. .SH "SEE ALSO" .PP ltfs(8), ltfsck(8), tape-backend(4), kmi-backend(4), ltfs.conf(5). diff --git a/man/sgml/mkltfs.sgml b/man/sgml/mkltfs.sgml index d0ed00ed..6304945a 100644 --- a/man/sgml/mkltfs.sgml +++ b/man/sgml/mkltfs.sgml @@ -228,6 +228,14 @@ + + + + + Use destructive format/unformat. This operation takes longer time in the LTO9 drive or later because of the media optimization procedure. + + + diff --git a/messages/bin_mkltfs/root.txt b/messages/bin_mkltfs/root.txt index 245bd7bd..7923a0de 100644 --- a/messages/bin_mkltfs/root.txt +++ b/messages/bin_mkltfs/root.txt @@ -127,5 +127,6 @@ root:table { 15422I:string { " --syslogtrace Enable diagnostic output to stderr and syslog" } 15423I:string { " -V, --version Version information" } 15424I:string { " --long-wipe Unformat the medium and erase any data on the tape by overwriting special data pattern.\n This operation takes over 3 hours. Once you start, you cannot interrupt it" } + 15425I:string { " --destructive Use destructive format/unformat. This operation takes longer time\n in the LTO9 drive or later because of the media optimization procedure" } } } diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index 4a9f757f..885448cf 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -831,7 +831,9 @@ v 17287I:string { "Making R/O mount from the location (%c, %llu)." } 17288I:string { "Detected the final indexes (IP: Gen = %llu, Pos = %llu) (DP: Gen = %llu, Pos = %llu)." } 17289I:string { "Skip parsing the final index on IP." } - 17290E:string { "UUID in the index does not match the label." } + 17290I:string { "Partitioning the medium with the destructive method." } + 17291I:string { "Unpartitioning the medium with the destructive method." } + 17292E:string { "UUID in the index does not match the label." } // For Debug 19999I:string { "%s %s %d." } diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 541a67cf..5dcb51cf 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1959,7 +1959,7 @@ int ltfs_mount_indexfile(char* filename, bool label_check, struct ltfs_volume *v * Volume UUID in label and it on index is not matched. * Actual UUID is not printed to avoid illegal modification by hand. */ - ltfsmsg(LTFS_ERR, 17290E); + ltfsmsg(LTFS_ERR, 17292E); ret = -LTFS_LABEL_MISMATCH; } } @@ -3008,9 +3008,12 @@ int ltfs_write_label(tape_partition_t partition, struct ltfs_volume *vol) * the device. * @param vol LTFS volume. The label structure receives a new UUID and format time; all other * label fields should be filled in correctly before calling this function. + * @param density_code density code to format. Used in IBM enterprise tape + * @param destructive use destructive format in MEDIUM_FORMAT command. The FORMAT field is + * specified to 2h. This might trigger the media optimization in LTO9 or later * @return 0 on success or a negative value on error. */ -int ltfs_format_tape(struct ltfs_volume *vol, int density_code) +int ltfs_format_tape(struct ltfs_volume *vol, int density_code, bool destructive) { int ret; struct tc_drive_param cart_param; @@ -3090,8 +3093,11 @@ int ltfs_format_tape(struct ltfs_volume *vol, int density_code) /* Format the tape */ INTERRUPTED_RETURN(); - ltfsmsg(LTFS_INFO, 11097I); - ret = tape_format(vol->device, ltfs_part_id2num(vol->label->partid_ip, vol), density_code); + if (destructive) + ltfsmsg(LTFS_INFO, 17290I); + else + ltfsmsg(LTFS_INFO, 11097I); + ret = tape_format(vol->device, ltfs_part_id2num(vol->label->partid_ip, vol), density_code, destructive); if (ret < 0) { ltfsmsg(LTFS_ERR, 11098E, ret); return ret; @@ -3153,9 +3159,11 @@ int ltfs_format_tape(struct ltfs_volume *vol, int density_code) * the device. * @param vol LTFS volume. * @param long_wipe invoke long erase after un-partitioning + * @param destructive use destructive unformat in MEDIUM_FORMAT command. The FORMAT field is + * specified to 2h. This might trigger the media optimization in LTO9 or later * @return 0 on success or a negative value on error. */ -int ltfs_unformat_tape(struct ltfs_volume *vol, bool long_wipe) +int ltfs_unformat_tape(struct ltfs_volume *vol, bool long_wipe, bool destructive) { int ret; @@ -3181,8 +3189,14 @@ int ltfs_unformat_tape(struct ltfs_volume *vol, bool long_wipe) /* Unformat the tape */ INTERRUPTED_RETURN(); - ltfsmsg(LTFS_INFO, 17071I); - ret = tape_unformat(vol->device); + if (destructive) { + ltfsmsg(LTFS_INFO, 17291I); + ret = tape_unformat_hard(vol->device); + } + else { + ltfsmsg(LTFS_INFO, 17071I); + ret = tape_unformat(vol->device); + } if (ret < 0) { ltfsmsg(LTFS_ERR, 17072E, ret); return ret; diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index 07b006aa..e34ffe3e 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -668,8 +668,8 @@ int ltfs_set_volume_name(const char *volname, struct ltfs_volume *vol); int ltfs_set_partition_map(char dp, char ip, int dp_num, int ip_num, struct ltfs_volume *vol); int ltfs_reset_capacity(bool reset, struct ltfs_volume *vol); int ltfs_write_label(tape_partition_t partition, struct ltfs_volume *vol); -int ltfs_format_tape(struct ltfs_volume *vol, int density_code); -int ltfs_unformat_tape(struct ltfs_volume *vol, bool long_erase); +int ltfs_format_tape(struct ltfs_volume *vol, int density_code, bool destructive); +int ltfs_unformat_tape(struct ltfs_volume *vol, bool long_erase, bool destructive); bool ltfs_is_dirty(struct ltfs_volume *vol); int ltfs_load_all_attributes(struct ltfs_volume *vol); diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index 892f1d6a..4564d312 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -1459,14 +1459,18 @@ static int tape_update_density(struct device_data *dev, int density_code) /** * Format tape for LTFS (make dual-partition tape) * @param dev device to format - * @param index_part partition number for index partition + * @param index_part partition number for index partition, unformat a tape when UINT32_MAX is specified + * @param hard use the hard format (Destroy all data and make 2 partition medium). It might take a + * longer time than soft format because of the media optimization feature + * (media calibration) in LTO9 drive or later. * @return 0 on success or a negative value on error */ -int tape_format(struct device_data *dev, tape_partition_t index_part, int density_code) +int tape_format(struct device_data *dev, tape_partition_t index_part, int density_code, bool hard) { int ret; unsigned char mp_medium_partition[TC_MP_MEDIUM_PARTITION_SIZE+4]; int page_length = TC_MP_MEDIUM_PARTITION_SIZE; + TC_FORMAT_TYPE format_type = TC_FORMAT_PARTITION; CHECK_ARG_NULL(dev, -LTFS_NULL_ARG); CHECK_ARG_NULL(dev->backend, -LTFS_NULL_ARG); @@ -1505,23 +1509,41 @@ int tape_format(struct device_data *dev, tape_partition_t index_part, int densit /* Set appropriate values to the page and Issue Mode Select */ mp_medium_partition[0] = 0x00; mp_medium_partition[1] = 0x00; - mp_medium_partition[19] = 0x01; - mp_medium_partition[20] = 0x20 | (mp_medium_partition[20] & 0x1F); /* Set FDP=0, SDP=0, IDP=1 ==> User Setting */ - mp_medium_partition[22] = 0x09; /* Set partition unit as gigabytes (10^9) */ - if (index_part == 1) { - mp_medium_partition[24] = 0xFF; /* Set Partition0 Capacity */ + + if (index_part == UINT32_MAX) { + mp_medium_partition[19] = 0x00; /* No additional partition, means one-partitioned */ + mp_medium_partition[20] = 0x20 | (mp_medium_partition[20] & 0x1F); /* Set FDP=0, SDP=0, IDP=1 ==> FIXED DATA PARTITION */ + + mp_medium_partition[22] = 0x08; + + mp_medium_partition[24] = 0xFF; mp_medium_partition[25] = 0xFF; - /* Set Partition1 Capacity to 1GB, This value will round up to minimum partition size in FCR3175-r2 */ - /* This field meaning will be chnaged in FCR3175-r3. In r3 n of "minumim partition size * n" should be specified. */ - /* If set this parameter to 1, we can support both specs. */ - /* In r2, this value is rounded up to minimum partition size. In r3, this value is the correct value.*/ - mp_medium_partition[26] = 0x00; /* Set Partition1 Capacity */ - mp_medium_partition[27] = 1; /* will round up to minimum partition size */ + mp_medium_partition[26] = 0x00; + mp_medium_partition[27] = 0x00; } else { - mp_medium_partition[24] = 0x00; /* Set Partition0 Capacity */ - mp_medium_partition[25] = 1; /* will round up to minimum partition size */ - mp_medium_partition[26] = 0xFF; /* Set Partition1 Capacity */ - mp_medium_partition[27] = 0xFF; + if (index_part == 1) { + mp_medium_partition[19] = 0x01; + mp_medium_partition[20] = 0x20 | (mp_medium_partition[20] & 0x1F); /* Set FDP=0, SDP=0, IDP=1 ==> User Setting */ + mp_medium_partition[22] = 0x09; /* Set partition unit as gigabytes (10^9) */ + + mp_medium_partition[24] = 0xFF; /* Set Partition0 Capacity */ + mp_medium_partition[25] = 0xFF; + /* Set Partition1 Capacity to 1GB, This value will round up to minimum partition size in FCR3175-r2 */ + /* This field meaning will be chnaged in FCR3175-r3. In r3 n of "minumim partition size * n" should be specified. */ + /* If set this parameter to 1, we can support both specs. */ + /* In r2, this value is rounded up to minimum partition size. In r3, this value is the correct value.*/ + mp_medium_partition[26] = 0x00; /* Set Partition1 Capacity */ + mp_medium_partition[27] = 1; /* will round up to minimum partition size */ + } else { + mp_medium_partition[19] = 0x01; + mp_medium_partition[20] = 0x20 | (mp_medium_partition[20] & 0x1F); /* Set FDP=0, SDP=0, IDP=1 ==> User Setting */ + mp_medium_partition[22] = 0x09; /* Set partition unit as gigabytes (10^9) */ + + mp_medium_partition[24] = 0x00; /* Set Partition0 Capacity */ + mp_medium_partition[25] = 1; /* will round up to minimum partition size */ + mp_medium_partition[26] = 0xFF; /* Set Partition1 Capacity */ + mp_medium_partition[27] = 0xFF; + } } if (mp_medium_partition[17] > 0x0A) { @@ -1534,8 +1556,11 @@ int tape_format(struct device_data *dev, tape_partition_t index_part, int densit page_length ); - /* Issue Format Medium (destroy all medium data and make 2-partitition medium) */ - ret = dev->backend->format(dev->backend_data, TC_FORMAT_DEST_PART, NULL, NULL, NULL); + /* Issue Format Medium (Make partitioned medium) */ + if (hard) + format_type = TC_FORMAT_DEST_PART; + + ret = dev->backend->format(dev->backend_data, format_type, NULL, NULL, NULL); if (ret < 0) { ltfsmsg(LTFS_ERR, 12053E, ret); return ret; @@ -1547,11 +1572,29 @@ int tape_format(struct device_data *dev, tape_partition_t index_part, int densit } /** - * Unformat tape (make single partition tape) + * Unformat tape (make single partition tape with hard method) + * LTO9 or later drive might triggers the media optimization feature (media calibration) + * when FORMAT_MEDIUM command is issued. It might take a long time to unformat. * @param dev device to format * @return 0 on success or a negative value on error */ int tape_unformat(struct device_data *dev) +{ + int ret; + + ret = tape_format(dev, UINT32_MAX, 0, false); + + return ret; +} + +/** + * Unformat tape (make single partition tape with hard method) + * LTO9 or later drive might triggers the media optimization feature (media calibration) + * when FORMAT_MEDIUM command is issued. It might take a long time to unformat. + * @param dev device to format + * @return 0 on success or a negative value on error + */ +int tape_unformat_hard(struct device_data *dev) { int ret; struct tc_position bom = { .partition = 0, .block = 0, .filemarks = 0 }; @@ -1582,8 +1625,7 @@ int tape_unformat(struct device_data *dev) } /* Reset partition space flag */ - dev->partition_space[0] = - dev->partition_space[1] = PART_WRITABLE; + dev->partition_space[0] = dev->partition_space[1] = PART_WRITABLE; return 0; } diff --git a/src/libltfs/tape.h b/src/libltfs/tape.h index 0901c091..9bfaf108 100644 --- a/src/libltfs/tape.h +++ b/src/libltfs/tape.h @@ -173,8 +173,9 @@ ssize_t tape_read(struct device_data *dev, char *buf, size_t count, const bool u int tape_erase(struct device_data *dev, bool long_erase); int tape_reset_capacity(struct device_data *dev); -int tape_format(struct device_data *dev, tape_partition_t index_part, int density_code); +int tape_format(struct device_data *dev, tape_partition_t index_part, int density_code, bool hard); int tape_unformat(struct device_data *dev); +int tape_unformat_hard(struct device_data *dev); ssize_t tape_write(struct device_data *dev, const char *buf, size_t count, bool ignore_less, bool ignore_nospc); int tape_write_filemark(struct device_data *dev, uint8_t count, bool ignore_less, bool ignore_nospc, bool immed); diff --git a/src/utils/mkltfs.c b/src/utils/mkltfs.c index 208f8e55..75b5a03d 100644 --- a/src/utils/mkltfs.c +++ b/src/utils/mkltfs.c @@ -105,7 +105,8 @@ struct other_format_opts { bool trace; /**< Debug mode indicator */ bool syslogtrace; /**< Generate debug output to stderr and syslog*/ bool fulltrace; /**< Full trace mode indicator */ - bool long_wipe; /**< Clean up whole tape by long erase? */ + bool long_wipe; /**< Clean up whole tape by long erase? */ + bool destructive; /**< Use destructive FORMAT_MEDIUM or not */ }; /* Forward declarations */ @@ -130,6 +131,7 @@ static struct option long_options[] = { {"keep-capacity", 0, 0, 'k'}, {"wipe", 0, 0, 'w'}, {"long-wipe", 0, 0, '+'}, + {"destructive", 0, 0, '&'}, {"force", 0, 0, 'f'}, {"quiet", 0, 0, 'q'}, {"trace", 0, 0, 't'}, @@ -182,6 +184,7 @@ void show_usage(char *appname, struct config_file *config, bool full) ltfsresult(15419I); /* -k, --keep-capacity */ ltfsresult(15417I); /* -x, --fulltrace */ ltfsresult(15424I); /* --long-wipe */ + ltfsresult(15425I); /* --destructive */ fprintf(stderr, "\n"); plugin_usage(appname, "driver", config); fprintf(stderr, "\n"); @@ -265,6 +268,7 @@ int main(int argc, char **argv) opt.quiet = false; opt.blocksize = LTFS_DEFAULT_BLOCKSIZE; opt.long_wipe = false; + opt.destructive = false; /* Check for a config file path given on the command line */ while (true) { @@ -342,6 +346,9 @@ int main(int argc, char **argv) opt.unformat = true; opt.long_wipe = true; break; + case '&': + opt.destructive = true; + break; case 'q': opt.quiet = true; break; @@ -669,7 +676,7 @@ int format_tape(struct ltfs_volume *vol, struct other_format_opts *opt, void *ar /* Create partitions and write labels and indices to the tape */ ltfsmsg(LTFS_INFO, 15010I, DATA_PART_ID, DATA_PART_NUM); ltfsmsg(LTFS_INFO, 15011I, INDEX_PART_ID, INDEX_PART_NUM); - ret = ltfs_format_tape(vol, 0); + ret = ltfs_format_tape(vol, 0, opt->destructive); if (ret < 0) { if (ret == -LTFS_INTERRUPTED) { ltfsmsg(LTFS_ERR, 15045E); @@ -814,7 +821,7 @@ int unformat_tape(struct ltfs_volume *vol, struct other_format_opts *opt, void * ltfsmsg(LTFS_DEBUG, 15007D); /* Create 1 partition cartridge */ - ret = ltfs_unformat_tape(vol, opt->long_wipe); + ret = ltfs_unformat_tape(vol, opt->long_wipe, opt->destructive); if (ret < 0) { if (ret == -LTFS_INTERRUPTED) { ltfsmsg(LTFS_ERR, 15046E); From a4cd6afd0e12c53a9429ecc1cd6dbbcdc28bc59f Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Sat, 19 Feb 2022 09:55:59 +0900 Subject: [PATCH 076/121] Export a couple of utility functions (#338) --- src/libltfs/xattr.c | 94 ++++++++++++++++++++++----------------------- src/libltfs/xattr.h | 5 +++ 2 files changed, 52 insertions(+), 47 deletions(-) diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index 3f188a95..40544105 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -148,29 +148,6 @@ static int _xattr_get_dentry_time(struct dentry *d, struct ltfs_timespec *val, c return ret; } -static int _xattr_get_string(const char *val, char **outval, const char *msg) -{ - if (! val) - return 0; - *outval = strdup(val); - if (! (*outval)) { - ltfsmsg(LTFS_ERR, 10001E, msg); - return -LTFS_NO_MEMORY; - } - return 0; -} - -static int _xattr_get_u64(uint64_t val, char **outval, const char *msg) -{ - int ret = asprintf(outval, "%"PRIu64, val); - if (ret < 0) { - ltfsmsg(LTFS_ERR, 10001E, msg); - *outval = NULL; - ret = -LTFS_NO_MEMORY; - } - return ret; -} - static int _xattr_get_tapepos(struct tape_offset *val, char **outval, const char *msg) { int ret = asprintf(outval, "%c:%"PRIu64, val->partition, val->block); @@ -593,24 +570,24 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons } else if (! strcmp(name, "ltfs.driveCaptureDump")) { ret = tape_takedump_drive(vol->device, true); } else if (! strcmp(name, "ltfs.fileUID")) { - ret = _xattr_get_u64(d->uid, &val, name); + ret = xattr_get_u64(d->uid, &val, name); } else if (! strcmp(name, "ltfs.volumeUUID")) { - ret = _xattr_get_string(vol->label->vol_uuid, &val, name); + ret = xattr_get_string(vol->label->vol_uuid, &val, name); } else if (! strcmp(name, "ltfs.volumeName")) { ltfs_mutex_lock(&vol->index->dirty_lock); - ret = _xattr_get_string(vol->index->volume_name.name, &val, name); + ret = xattr_get_string(vol->index->volume_name.name, &val, name); ltfs_mutex_unlock(&vol->index->dirty_lock); } else if (! strcmp(name, "ltfs.softwareVersion")) { - ret = _xattr_get_string(PACKAGE_VERSION, &val, name); + ret = xattr_get_string(PACKAGE_VERSION, &val, name); } else if (! strcmp(name, "ltfs.softwareFormatSpec")) { - ret = _xattr_get_string(LTFS_INDEX_VERSION_STR, &val, name); + ret = xattr_get_string(LTFS_INDEX_VERSION_STR, &val, name); } else if (! strcmp(name, "ltfs.softwareVendor")) { - ret = _xattr_get_string(LTFS_VENDOR_NAME, &val, name); + ret = xattr_get_string(LTFS_VENDOR_NAME, &val, name); } else if (! strcmp(name, "ltfs.softwareProduct")) { if ( strncmp( PACKAGE_VERSION, "1", 1 )==0 ) - ret = _xattr_get_string("LTFS SDE", &val, name); + ret = xattr_get_string("LTFS SDE", &val, name); else if ( strncmp( PACKAGE_VERSION, "2", 1 )==0 ) - ret = _xattr_get_string("LTFS LE", &val, name); + ret = xattr_get_string("LTFS LE", &val, name); else ret = -LTFS_NO_XATTR; } else if (! strcmp(name, "ltfs.vendor.IBM.logLevel")) { @@ -707,7 +684,7 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons val[1] = '\0'; } } else if (! strcmp(name, "ltfs.startblock")) { - ret = _xattr_get_u64(TAILQ_FIRST(&d->extentlist)->start.block, &val, name); + ret = xattr_get_u64(TAILQ_FIRST(&d->extentlist)->start.block, &val, name); } } @@ -715,10 +692,10 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons if (ret == -LTFS_NO_XATTR && d == vol->index->root) { if (! strcmp(name, "ltfs.commitMessage")) { ltfs_mutex_lock(&vol->index->dirty_lock); - ret = _xattr_get_string(vol->index->commit_message, &val, name); + ret = xattr_get_string(vol->index->commit_message, &val, name); ltfs_mutex_unlock(&vol->index->dirty_lock); } else if (! strcmp(name, "ltfs.volumeSerial")) { - ret = _xattr_get_string(vol->label->barcode, &val, name); + ret = xattr_get_string(vol->label->barcode, &val, name); } else if (! strcmp(name, "ltfs.volumeFormatTime")) { ret = _xattr_get_time(&vol->label->format_time, &val, name); if (ret == LTFS_TIME_OUT_OF_RANGE) { @@ -726,9 +703,9 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons ret = 0; } } else if (! strcmp(name, "ltfs.volumeBlocksize")) { - ret = _xattr_get_u64(vol->label->blocksize, &val, name); + ret = xattr_get_u64(vol->label->blocksize, &val, name); } else if (! strcmp(name, "ltfs.indexGeneration")) { - ret = _xattr_get_u64(vol->index->generation, &val, name); + ret = xattr_get_u64(vol->index->generation, &val, name); } else if (! strcmp(name, "ltfs.indexTime")) { ret = _xattr_get_time(&vol->index->mod_time, &val, name); if (ret == LTFS_TIME_OUT_OF_RANGE) { @@ -736,22 +713,22 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons ret = 0; } } else if (! strcmp(name, "ltfs.policyExists")) { - ret = _xattr_get_string(ic->have_criteria ? "true" : "false", &val, name); + ret = xattr_get_string(ic->have_criteria ? "true" : "false", &val, name); } else if (! strcmp(name, "ltfs.policyAllowUpdate")) { - ret = _xattr_get_string(vol->index->criteria_allow_update ? "true" : "false", + ret = xattr_get_string(vol->index->criteria_allow_update ? "true" : "false", &val, name); } else if (! strcmp(name, "ltfs.policyMaxFileSize") && ic->have_criteria) { - ret = _xattr_get_u64(ic->max_filesize_criteria, &val, name); + ret = xattr_get_u64(ic->max_filesize_criteria, &val, name); } else if (! strcmp(name, "ltfs.volumeCompression")) { - ret = _xattr_get_string(vol->label->enable_compression ? "true" : "false", &val, name); + ret = xattr_get_string(vol->label->enable_compression ? "true" : "false", &val, name); } else if (! strcmp(name, "ltfs.indexLocation")) { ret = _xattr_get_tapepos(&vol->index->selfptr, &val, name); } else if (! strcmp(name, "ltfs.indexPrevious")) { ret = _xattr_get_tapepos(&vol->index->backptr, &val, name); } else if (! strcmp(name, "ltfs.indexCreator")) { - ret = _xattr_get_string(vol->index->creator, &val, name); + ret = xattr_get_string(vol->index->creator, &val, name); } else if (! strcmp(name, "ltfs.labelCreator")) { - ret = _xattr_get_string(vol->label->creator, &val, name); + ret = xattr_get_string(vol->label->creator, &val, name); } else if (! strcmp(name, "ltfs.indexVersion")) { ltfs_mutex_lock(&vol->index->dirty_lock); ret = _xattr_get_version(vol->index->version, &val, name); @@ -809,18 +786,18 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons } else if (! strcmp(name, "ltfs.mediaIndexPartitionAvailableSpace")) { ret = _xattr_get_cartridge_capacity(&cap, &cap.remaining_ip, &val, name, vol); } else if (! strcmp(name, "ltfs.mediaEncrypted")) { - ret = _xattr_get_string(tape_get_media_encrypted(vol->device), &val, name); + ret = xattr_get_string(tape_get_media_encrypted(vol->device), &val, name); } else if (! strcmp(name, "ltfs.mediaPool.additionalInfo")) { char *tmp=NULL; ret = tape_get_media_pool_info(vol, &tmp, &val); if (ret < 0 || !val) ret = -LTFS_NO_XATTR; } else if (! strcmp(name, "ltfs.driveEncryptionState")) { - ret = _xattr_get_string(tape_get_drive_encryption_state(vol->device), &val, name); + ret = xattr_get_string(tape_get_drive_encryption_state(vol->device), &val, name); } else if (! strcmp(name, "ltfs.driveEncryptionMethod")) { - ret = _xattr_get_string(tape_get_drive_encryption_method(vol->device), &val, name); + ret = xattr_get_string(tape_get_drive_encryption_method(vol->device), &val, name); } else if (! strcmp(name, "ltfs.vendor.IBM.referencedBlocks")) { - ret = _xattr_get_u64(ltfs_get_valid_block_count_unlocked(vol), &val, name); + ret = xattr_get_u64(ltfs_get_valid_block_count_unlocked(vol), &val, name); } else if (! strcmp(name, "ltfs.vendor.IBM.trace")) { ret = ltfs_get_trace_status(&val); } else if (! strcmp(name, "ltfs.vendor.IBM.totalBlocks")) { @@ -828,7 +805,7 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons if (ret < 0) val = NULL; else - ret = _xattr_get_u64(append_pos, &val, name); + ret = xattr_get_u64(append_pos, &val, name); } else if (! strcmp(name, "ltfs.vendor.IBM.cartridgeMountNode")) { ret = asprintf(&val, "localhost"); if (ret < 0) { @@ -1295,6 +1272,29 @@ static int _xattr_remove_virtual(struct dentry *d, const char *name, struct ltfs /* Global functions */ +int xattr_get_string(const char *val, char **outval, const char *msg) +{ + if (! val) + return 0; + *outval = strdup(val); + if (! (*outval)) { + ltfsmsg(LTFS_ERR, 10001E, msg); + return -LTFS_NO_MEMORY; + } + return 0; +} + +int xattr_get_u64(uint64_t val, char **outval, const char *msg) +{ + int ret = asprintf(outval, "%"PRIu64, val); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 10001E, msg); + *outval = NULL; + ret = -LTFS_NO_MEMORY; + } + return ret; +} + int xattr_do_set(struct dentry *d, const char *name, const char *value, size_t size, struct xattr_info *xattr) { diff --git a/src/libltfs/xattr.h b/src/libltfs/xattr.h index 2a47568d..63cd1070 100644 --- a/src/libltfs/xattr.h +++ b/src/libltfs/xattr.h @@ -70,6 +70,11 @@ extern "C" { #define LTFS_PRIVATE_PREFIX "ltfs." +/* Explicitly export functions as utility */ +int xattr_get_string(const char *val, char **outval, const char *msg); +int xattr_get_u64(uint64_t val, char **outval, const char *msg); + +/* Official functions to export */ int xattr_set(struct dentry *d, const char *name, const char *value, size_t size, int flags, struct ltfs_volume *vol); int xattr_get(struct dentry *d, const char *name, char *value, size_t size, From 98c6c6beceb9ced1be9b74016d1d5f9775ecc2b7 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 21 Feb 2022 17:03:52 +0900 Subject: [PATCH 077/121] Move to Rocky Linux --- .github/workflows/build-centos8.yml | 4 +-- README.md | 44 ++++++++++++++--------------- cd | 0 3 files changed, 24 insertions(+), 24 deletions(-) create mode 100644 cd diff --git a/.github/workflows/build-centos8.yml b/.github/workflows/build-centos8.yml index f946a4f6..a33e1be6 100644 --- a/.github/workflows/build-centos8.yml +++ b/.github/workflows/build-centos8.yml @@ -4,7 +4,7 @@ on: [push, pull_request] jobs: build: - name: Build on CentOS8 + name: Build on CentOS8 (Rocky Linux) runs-on: ubuntu-latest steps: @@ -12,6 +12,6 @@ jobs: uses: actions/checkout@v1 - name: Build LTFS id: build - uses: LinearTapeFileSystem/CentOS8-Build@v1.4 + uses: LinearTapeFileSystem/CentOS8-Build@v1.5 with: destination: '/tmp/ltfs' diff --git a/README.md b/README.md index 4bfbff5a..c24013c5 100644 --- a/README.md +++ b/README.md @@ -31,12 +31,12 @@ At this time, the target of this project to meet is the [LTFS format specificati | HP | LTO6 | T.B.D. | | HP | LTO7 | T.B.D. | | HP | LTO8 | T.B.D. | - | HP | LTO9 | T.B.D. | + | HP | LTO9 | T.B.D. | | Quantum | LTO5 (Only Half Height) | T.B.D. | | Quantum | LTO6 (Only Half Height) | T.B.D. | | Quantum | LTO7 (Only Half Height) | T.B.D. | | Quantum | LTO8 (Only Half Height) | T.B.D. | - | Quantum | LTO9 (Only Half Height) | T.B.D. | + | Quantum | LTO9 (Only Half Height) | T.B.D. | ## LTFS Format Specifications @@ -137,26 +137,26 @@ In some systems, you might need `sudo ldconfig -v` after `make install` to load ### Buildable Linux distributions - | Dist | Arch | Status | - |:----------------------------------:|:-------:|:-------------------------------------------------------------------------------------------------------------------------------------:| - | RHEL 8 | x86\_64 | OK - Not checked automatically | - | RHEL 8 | ppc64le | OK - Not checked automatically | - | RHEL 7 | x86\_64 | OK - Not checked automatically | - | RHEL 7 | ppc64le | OK - Not checked automatically | - | CentOS 8 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS8%20Build%20Job/badge.svg?branch=master) | - | CentOS 8 | ppc64le | OK - Not checked automatically | - | CentOS 7 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS7%20Build%20Job/badge.svg?branch=master) | - | CentOS 7 | ppc64le | OK - Not checked automatically | - | Fedora 28 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Fedora28%20Build%20Job/badge.svg?branch=master) | - | Ubuntu 16.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2016.04%20Build%20Job/badge.svg?branch=master) | - | Ubuntu 16.04 LTS | ppc64le | OK - Not checked automatically | - | Ubuntu 18.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2018.04%20Build%20Job/badge.svg?branch=master) | - | Ubuntu 18.04 LTS | ppc64le | OK - Not checked automatically | - | Ubuntu 20.04 LTS (Need icu-config) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2020.04%20Build%20Job/badge.svg?branch=master) | - | Debian 9 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian9%20Build%20Job/badge.svg?branch=master) | - | Debian 10 (Need icu-config) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian10%20Build%20Job/badge.svg?branch=master) | - | ArchLinux 2018.08.01 | x86\_64 | OK - Not checked automatically | - | ArchLinux 2018.12.31 (rolling) | x86\_64 | OK - Not checked automatically | + | Dist | Arch | Status | + |:----------------------------------:|:-------:|:--------------------------------------------------------------------------------------------------------------------------------:| + | RHEL 8 | x86\_64 | OK - Not checked automatically | + | RHEL 8 | ppc64le | OK - Not checked automatically | + | RHEL 7 | x86\_64 | OK - Not checked automatically | + | RHEL 7 | ppc64le | OK - Not checked automatically | + | CentOS 8 (Rocky Linux) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS8%20Build%20Job/badge.svg?branch=master) | + | CentOS 8 (Rocky Linux) | ppc64le | OK - Not checked automatically | + | CentOS 7 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS7%20Build%20Job/badge.svg?branch=master) | + | CentOS 7 | ppc64le | OK - Not checked automatically | + | Fedora 28 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Fedora28%20Build%20Job/badge.svg?branch=master) | + | Ubuntu 16.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2016.04%20Build%20Job/badge.svg?branch=master) | + | Ubuntu 16.04 LTS | ppc64le | OK - Not checked automatically | + | Ubuntu 18.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2018.04%20Build%20Job/badge.svg?branch=master) | + | Ubuntu 18.04 LTS | ppc64le | OK - Not checked automatically | + | Ubuntu 20.04 LTS (Need icu-config) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2020.04%20Build%20Job/badge.svg?branch=master) | + | Debian 9 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian9%20Build%20Job/badge.svg?branch=master) | + | Debian 10 (Need icu-config) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Debian10%20Build%20Job/badge.svg?branch=master) | + | ArchLinux 2018.08.01 | x86\_64 | OK - Not checked automatically | + | ArchLinux 2018.12.31 (rolling) | x86\_64 | OK - Not checked automatically | Currently, automatic build checking is working on GitHub Actions and Travis CI. diff --git a/cd b/cd new file mode 100644 index 00000000..e69de29b From ec78f5dcda46b0e2fa88d26d700756e9a11215fa Mon Sep 17 00:00:00 2001 From: Chris Dinneen Date: Tue, 22 Feb 2022 09:10:50 +0800 Subject: [PATCH 078/121] Fix the description of the capture index option in man page of ltfs (#337) --- man/ltfs.8 | 6 +++--- man/sgml/ltfs.sgml | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/man/ltfs.8 b/man/ltfs.8 index 3f435c25..e159333e 100644 --- a/man/ltfs.8 +++ b/man/ltfs.8 @@ -1,10 +1,10 @@ .\" auto-generated by docbook2man-spec from docbook-utils package -.TH "LTFS" "8" "20 January 2022" "LTFS" "LTFS Command Reference" +.TH "LTFS" "8" "21 February 2022" "LTFS" "LTFS Command Reference" .SH NAME ltfs \- File system based on a linear tape drive .SH SYNOPSIS .sp -\fBltfs\fR \fB\fImount_point\fB\fR [ \fB-o \fImount_option\fB\fR\fI...\fR ] [ \fB-V\fR ] [ \fB-h\fR ] [ \fB-a\fR ] +\fBltfs\fR \fB\fImount_point\fB\fR [ \fB-o \fImount_option\fB\fR\fI...\fR ] [ \fB-V\fR ] [ \fB-h\fR ] [ \fB-a\fR ] .SH "DESCRIPTION" .PP \fBltfs\fR is a file system for linear tape drive. @@ -182,7 +182,7 @@ Override output verbosity directly (default: 2) \fB-o noeject\fR Do not eject the cartridge after unmount (default) .TP -\fB--capture-index=\fIdir\fB\fR +\fB-o capture_index=\fIdir\fB\fR Capture index to the specified directory by dir when index is updated. File name of each index is [BARCODE]-[GEN]-[PARTITION].xml if tape serial (barcode) is specified at format time. Otherwise it is [VOL_UUID]-[GEN]-[PARTITION].xml. diff --git a/man/sgml/ltfs.sgml b/man/sgml/ltfs.sgml index 077dcd66..9d754ac2 100644 --- a/man/sgml/ltfs.sgml +++ b/man/sgml/ltfs.sgml @@ -359,7 +359,7 @@ - + Capture index to the specified directory by dir when index is updated. From e62d21c97d36924c70f68f28fa1217741e773620 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 28 Feb 2022 11:35:15 +0900 Subject: [PATCH 079/121] Fix link breakages in README (#340) --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index c24013c5..cc241ee7 100644 --- a/README.md +++ b/README.md @@ -44,12 +44,12 @@ LTFS Format Specification is specified data placement, shape of index and names The table below show status of the LTFS format Specification - | Version | Status of SNIA | Status of ISO | - |:-------:|:------------------------------------------------------------------------------------------------------------:|:-------------------------:| - | 2.2 | [Published](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | Published as `20919:2016` | - | 2.3.1 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.PDF) | - | - | 2.4 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | - | - | 2.5.1 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2%205%201_Standard.pdf) | Published as `20919:2021` | + | Version | Status of SNIA | Status of ISO | + |:-------:|:---------------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------:| + | 2.2 | [Published](http://snia.org/sites/default/files/LTFS_Format_2.2.0_Technical_Position.pdf) | [Published as `20919:2016`](https://www.iso.org/standard/69458.html) | + | 2.3.1 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.3.1_TechPosition.PDF) | - | + | 2.4 | [Published](https://www.snia.org/sites/default/files/technical_work/LTFS/LTFS_Format_2.4.0_TechPosition.pdf) | - | + | 2.5.1 | [Published](https://www.snia.org/sites/default/files/technical-work/ltfs/release/SNIA-LTFS-Format-2-5-1-Standard.pdf) | [Published as `20919:2021`](https://www.iso.org/standard/80598.html) | # How to use the LTFS (Quick start) From af2aeabc5e20daccf3eaf6fc218dbdbc418625d2 Mon Sep 17 00:00:00 2001 From: "Qays H. Poonawala" Date: Thu, 10 Mar 2022 18:16:52 -0800 Subject: [PATCH 080/121] Fix typos of homebrew in readme (#342) --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index cc241ee7..208329ba 100644 --- a/README.md +++ b/README.md @@ -164,7 +164,7 @@ For Ubuntu20.04 and Debian10, dummy `icu-config` is needed in the build machine. ## Build and install on OSX (macOS) -### Recent Homedrew system setup +### Recent Homebrew system setup Before build on macOS, you need to configure the environment like below. @@ -175,7 +175,7 @@ export PKG_CONFIG_PATH="/usr/local/opt/icu4c/lib/pkgconfig:/usr/local/opt/libxml export PATH="$PATH:$ICU_PATH:$LIBXML2_PATH" ``` -### Old Homedrew system setup +### Old Homebrew system setup Before build on OSX (macOS), some include path adjustment is required. ``` From ffc63ac64d511e683f88bb163c774b603366a684 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Tue, 8 Mar 2022 22:51:58 +0900 Subject: [PATCH 081/121] Fix a crash at parsing an extended attribute (#341) --- src/libltfs/xml_reader_libltfs.c | 75 +++++++++++++++++--------------- 1 file changed, 40 insertions(+), 35 deletions(-) diff --git a/src/libltfs/xml_reader_libltfs.c b/src/libltfs/xml_reader_libltfs.c index a134c4a9..c4efa03f 100644 --- a/src/libltfs/xml_reader_libltfs.c +++ b/src/libltfs/xml_reader_libltfs.c @@ -772,7 +772,7 @@ static int _xml_parse_extents(xmlTextReaderPtr reader, int idx_version, struct d static int _xml_parse_one_xattr(xmlTextReaderPtr reader, struct dentry *d) { char *xattr_type; - struct xattr_info *xattr; + struct xattr_info *xattr = NULL; declare_parser_vars("xattr"); declare_tracking_arrays(2, 0); @@ -793,6 +793,7 @@ static int _xml_parse_one_xattr(xmlTextReaderPtr reader, struct dentry *d) if (ret < 0) { ltfsmsg(LTFS_WARN, 17269W, d->name.name); free(xattr); + xattr = NULL; } check_tag_end("key"); @@ -808,53 +809,57 @@ static int _xml_parse_one_xattr(xmlTextReaderPtr reader, struct dentry *d) } check_empty(); - if (empty == 0) { - ret = xml_scan_text(reader, &value); - if (ret < 0) { - free(xattr->key.name); - free(xattr); - return ret; - } - - if (! xattr_type || ! strcmp(xattr_type, "text")) { - xattr->value = strdup(value); - if (! xattr->value) { - ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + if (xattr) { + if (empty == 0) { + ret = xml_scan_text(reader, &value); + if (ret < 0) { free(xattr->key.name); free(xattr); - return -LTFS_NO_MEMORY; + return ret; } - xattr->size = strlen(value); - } else { /* base64 */ - xattr->size = base64_decode((const unsigned char *)value, strlen(value), - (unsigned char **)(&xattr->value)); - if (xattr->size == 0) { - ltfsmsg(LTFS_ERR, 17028E); - free(xattr->key.name); - free(xattr); - return -LTFS_XML_XATTR_SIZE; + + if (! xattr_type || ! strcmp(xattr_type, "text")) { + xattr->value = strdup(value); + if (! xattr->value) { + ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + free(xattr->key.name); + free(xattr); + return -LTFS_NO_MEMORY; + } + xattr->size = strlen(value); + } else { /* base64 */ + xattr->size = base64_decode((const unsigned char *)value, strlen(value), + (unsigned char **)(&xattr->value)); + if (xattr->size == 0) { + ltfsmsg(LTFS_ERR, 17028E); + free(xattr->key.name); + free(xattr); + return -LTFS_XML_XATTR_SIZE; + } } + if (strlen(value) > 0) + check_tag_end("value"); + } else { + xattr->value = NULL; + xattr->size = 0; } - if (strlen(value) > 0) - check_tag_end("value"); - } else { - xattr->value = NULL; - xattr->size = 0; } free(xattr_type); - } else ignore_unrecognized_tag(); } check_required_tags(); - TAILQ_INSERT_TAIL(&d->xattrlist, xattr, list); - if (!strcmp(xattr->key.name, "ltfs.vendor.IBM.immutable") && !strcmp(xattr->value, "1") ) { - d->is_immutable = true; - } - if (!strcmp(xattr->key.name, "ltfs.vendor.IBM.appendonly") && !strcmp(xattr->value, "1") ) { - d->is_appendonly = true; + if (xattr) { + TAILQ_INSERT_TAIL(&d->xattrlist, xattr, list); + + if (!strcmp(xattr->key.name, "ltfs.vendor.IBM.immutable") && !strcmp(xattr->value, "1") ) { + d->is_immutable = true; + } + if (!strcmp(xattr->key.name, "ltfs.vendor.IBM.appendonly") && !strcmp(xattr->value, "1") ) { + d->is_appendonly = true; + } } return 0; From 3919f56c0f3b422f8000c645d203b52b15a022bb Mon Sep 17 00:00:00 2001 From: manu0401 Date: Fri, 25 Mar 2022 01:55:49 +0100 Subject: [PATCH 082/121] Silence warnings from autoconf 2.70 and higher (#344) --- autogen.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/autogen.sh b/autogen.sh index 504a9748..113fb3f2 100755 --- a/autogen.sh +++ b/autogen.sh @@ -46,6 +46,6 @@ else libtoolize fi -autoconf -autoheader -automake --add-missing --copy +autoconf || echo "Ignore warning" +autoheader || echo "Ignore warning" +automake --add-missing --copy || echo "Ignore warning" From 7fe537dae99308896380666b28978a5b8952ae45 Mon Sep 17 00:00:00 2001 From: manu0401 Date: Fri, 25 Mar 2022 01:58:42 +0100 Subject: [PATCH 083/121] Fix undefined symbol error on NetBSD/amd64 (#346) --- messages/make_message_src.sh | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/messages/make_message_src.sh b/messages/make_message_src.sh index 82488e81..13ca3861 100755 --- a/messages/make_message_src.sh +++ b/messages/make_message_src.sh @@ -78,18 +78,20 @@ make_obj() { NetBSD) # generate libtool archive for later linking mv lib${BASENAME}.a ../../lib${BASENAME}_dat.a - OBJFILE=${BASENAME}_dat.o - LTFILE=${BASENAME}_dat.lo mkdir -p .libs ../../.libs - mv ${OBJFILE} .libs - LTVERS=`libtool --version | - sed -e 's/^\([^ ]*\) (GNU \(.*\)) \(.*\)$$/\1 - GNU \2 \3/' -e q` - echo "# ${OBJFILE} - a libtool object file" > ${LTFILE} - echo "# Generated by ${LTVERS}" >> ${LTFILE} - echo "pic_object='.libs/${OBJFILE}'" >> ${LTFILE} - echo "non_pic_object=none" >> ${LTFILE} - libtool --mode=link --tag=CC cc -o ../../lib${BASENAME}_dat.la ${LTFILE} - cp ../../lib${BASENAME}_dat.a ../../.libs + LTFILES="" + LTVERS=`libtool --version | + sed -e 's/^\([^ ]*\) (GNU \(.*\)) \(.*\)$$/\1 - GNU \2 \3/' -e q` + for OBJFILE in *.o ; do + LTFILE=${OBJFILE%.o}.lo + mv ${OBJFILE} .libs + echo "# ${OBJFILE} - a libtool object file" > ${LTFILE} + echo "# Generated by ${LTVERS}" >> ${LTFILE} + echo "pic_object='.libs/${OBJFILE}'" >> ${LTFILE} + echo "non_pic_object=none" >> ${LTFILE} + LTFILES="${LTFILES} ${LTFILE}" + done + libtool --mode=link --tag=CC cc -o ../../lib${BASENAME}_dat.la ${LTFILES} ;; *) mv ${BASENAME}_dat.o ../../lib${BASENAME}_dat.a From 98e8854e43fbd99323082737b726afa6cbb8c9d8 Mon Sep 17 00:00:00 2001 From: manu0401 Date: Fri, 25 Mar 2022 02:00:12 +0100 Subject: [PATCH 084/121] Fix filehandle corruption on LP64 (#345) Do not take for granted that __WORDSIZE is defined to distinguish between 32 and 64 bit platforms. If undefined on LP64 machine, STRUCT_TO_FILEHANDLE corrupts the filehande. Instead use ULONG_MAX from which is mandated by ISO C99. --- src/ltfs_fuse.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/ltfs_fuse.c b/src/ltfs_fuse.c index 00254728..e1347540 100644 --- a/src/ltfs_fuse.c +++ b/src/ltfs_fuse.c @@ -54,6 +54,9 @@ ** ************************************************************************************* */ + +#include /* for ULONG_MAX */ + #include "ltfs_fuse.h" #include "libltfs/ltfs_fsops.h" #include "libltfs/iosched.h" @@ -68,7 +71,7 @@ #include "libltfs/arch/win/win_util.h" #endif -#if (__WORDSIZE == 64) +#if (__WORDSIZE == 64 || ULONG_MAX == 0xffffffffffffffffUL) #define FILEHANDLE_TO_STRUCT(fh) ((struct ltfs_file_handle *)(uint64_t)(fh)) #define STRUCT_TO_FILEHANDLE(de) ((uint64_t)(de)) #else From 3b26f034e11a2a0cb767dfd914597b9a3527ffdc Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 6 Apr 2022 18:07:14 +0900 Subject: [PATCH 085/121] Return an error without issuing GRAO command in LTO9HH (#348) --- src/tape_drivers/linux/sg/sg_tape.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index be06b55f..2b950d08 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -5040,8 +5040,9 @@ int sg_grao(void *device, unsigned char *buf, const uint32_t len) if (IS_LTO(priv->drive_type)) { if ( DRIVE_GEN(priv->drive_type) == 0x05 || DRIVE_GEN(priv->drive_type) == 0x06 || - DRIVE_GEN(priv->drive_type) == 0x07 || DRIVE_GEN(priv->drive_type) == 0x08 ) { - /* LTO6-LTO8 don't support RAO commands */ + DRIVE_GEN(priv->drive_type) == 0x07 || DRIVE_GEN(priv->drive_type) == 0x08 || + DRIVE_GEN(priv->drive_type) == DRIVE_LTO9_HH ) { + /* LTO5-LTO8 and LTO9-HH don't support RAO commands */ return -EDEV_UNSUPPORETD_COMMAND; } } From b8e7384d294cb92e474957b8353e633e250a795c Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 10 Aug 2022 10:28:39 +0900 Subject: [PATCH 086/121] Improve error position detention at WRITE PERM (#357) * Correct miscalculation of last block on tape * Consider the final index on the partition as error position even if very small number is returned * Never adjust the force_writeperm threshold for better debug * Stop checking the I/F of the ltfs-backends repository --- .github/workflows/build-centos7.yml | 2 +- .github/workflows/build-centos8.yml | 2 +- messages/iosched_unified/root.txt | 3 ++- messages/libltfs/root.txt | 5 +++-- src/iosched/unified.c | 18 +++++++++++++-- src/libltfs/ltfs.c | 2 +- src/libltfs/ltfs_fsops_raw.c | 22 +++++++++++++++++-- src/libltfs/tape.c | 16 ++++++++------ src/libltfs/tape.h | 2 +- src/libltfs/tape_ops.h | 6 ++--- src/tape_drivers/freebsd/cam/cam_tc.c | 13 ++++++----- src/tape_drivers/generic/file/filedebug_tc.c | 16 +++++++++----- src/tape_drivers/generic/itdtimg/itdtimg_tc.c | 8 +++---- .../linux/lin_tape/lin_tape_ibmtape.c | 11 +++++----- src/tape_drivers/linux/sg/sg_tape.c | 18 +++++++++------ .../netbsd/scsipi-ibmtape/scsipi_ibmtape.c | 18 +++++++++------ src/tape_drivers/osx/iokit/iokit_tape.c | 19 ++++++++++------ src/tape_drivers/tape_drivers.h | 3 ++- 18 files changed, 120 insertions(+), 64 deletions(-) diff --git a/.github/workflows/build-centos7.yml b/.github/workflows/build-centos7.yml index 6fd8ddbd..2cdcce57 100644 --- a/.github/workflows/build-centos7.yml +++ b/.github/workflows/build-centos7.yml @@ -12,6 +12,6 @@ jobs: uses: actions/checkout@v1 - name: Build LTFS id: build - uses: LinearTapeFileSystem/CentOS7-Build@v1.3 + uses: LinearTapeFileSystem/CentOS7-Build@v1.4 with: destination: '/tmp/ltfs' diff --git a/.github/workflows/build-centos8.yml b/.github/workflows/build-centos8.yml index a33e1be6..c2080283 100644 --- a/.github/workflows/build-centos8.yml +++ b/.github/workflows/build-centos8.yml @@ -12,6 +12,6 @@ jobs: uses: actions/checkout@v1 - name: Build LTFS id: build - uses: LinearTapeFileSystem/CentOS8-Build@v1.5 + uses: LinearTapeFileSystem/CentOS8-Build@v1.6 with: destination: '/tmp/ltfs' diff --git a/messages/iosched_unified/root.txt b/messages/iosched_unified/root.txt index 3d8f4e0c..a2ecb64e 100644 --- a/messages/iosched_unified/root.txt +++ b/messages/iosched_unified/root.txt @@ -62,7 +62,8 @@ root:table { 13022W:string { "Freeing a dentry priv with outstanding write requests. This is a bug." } //unused 13023W:string { "Failed to copy a request: will stop writing file to the index partition." } 13024I:string { "Clean up extents and append index at index partition (%d)." } - 13025I:string { "Get error position (%d, %d)." } + 13025I:string { "Truncate extents larger than position (%d, %lld), block size = %ld." } 13026E:string { "Write perm handling error : %s (%d)." } + 13027I:string { "Error position is larger than last index position: (%d, %lld), last index = %lld." } } } diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index 885448cf..938f8811 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -429,7 +429,7 @@ root:table { 11332I:string { "Load successful." } 11333I:string { "A cartridge with write-perm error is detected on %s. Seek the newest index (IP: Gen = %llu, VCR = %llu) (DP: Gen = %llu, VCR = %llu) (VCR = %llu)." } 11334I:string { "Remove extent : %s (%llu, %llu)." } - 11335D:string { "Get physical block position (%d - %d)." } + //unused 11335D:string { "Get physical block position (%d - %d)." } 11336I:string { "The attribute does not exist. Ignore the expected error." } 11337I:string { "Update index-dirty flag (%d) - %s (0x%p)." } 11338I:string { "Syncing index of %s %s." } @@ -833,7 +833,8 @@ v 17289I:string { "Skip parsing the final index on IP." } 17290I:string { "Partitioning the medium with the destructive method." } 17291I:string { "Unpartitioning the medium with the destructive method." } - 17292E:string { "UUID in the index does not match the label." } + 17292I:string { "Current position is (%llu, %llu), Error position is (%llu, %llu)." } + 17293E:string { "UUID in the index does not match the label." } // For Debug 19999I:string { "%s %s %d." } diff --git a/src/iosched/unified.c b/src/iosched/unified.c index 56a39a04..43cdacd2 100644 --- a/src/iosched/unified.c +++ b/src/iosched/unified.c @@ -2250,6 +2250,7 @@ int _unified_write_index_after_perm(int write_ret, struct unified_data *priv) { int ret = 0; struct tc_position err_pos; + uint64_t last_index_pos = UINT64_MAX; unsigned long blocksize; if (!IS_WRITE_PERM(-write_ret)) { @@ -2263,14 +2264,27 @@ int _unified_write_index_after_perm(int write_ret, struct unified_data *priv) ltfsmsg(LTFS_ERR, 13026E, "update MAM", ret); blocksize = ltfs_get_blocksize(priv->vol); - ret = tape_get_physical_block_position(priv->vol->device, &err_pos); + + ret = tape_get_first_untransfered_position(priv->vol->device, &err_pos); if (ret < 0) { ltfsmsg(LTFS_ERR, 13026E, "get error pos", ret); return ret; } - ltfsmsg(LTFS_INFO, 13025I, (int)err_pos.block, (int)blocksize); + /* Check the err_pos is larger than the last index position of the partition */ + if (err_pos.partition == ltfs_part_id2num(priv->vol->label->partid_ip, priv->vol)) { + last_index_pos = priv->vol->ip_coh.set_id; + } else { + last_index_pos = priv->vol->dp_coh.set_id; + } + + if (last_index_pos > err_pos.block) { + ltfsmsg(LTFS_INFO, 13027I, (int)err_pos.partition, + (unsigned long long)err_pos.block, (unsigned long long)last_index_pos); + err_pos.block = last_index_pos + 1; + } + ltfsmsg(LTFS_INFO, 13025I, (int)err_pos.partition, (unsigned long long)err_pos.block, blocksize); ret = ltfs_fsraw_cleanup_extent(priv->vol->index->root, err_pos, blocksize, priv->vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 13026E, "extent cleanup", ret); diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 5dcb51cf..87a2cdf2 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1959,7 +1959,7 @@ int ltfs_mount_indexfile(char* filename, bool label_check, struct ltfs_volume *v * Volume UUID in label and it on index is not matched. * Actual UUID is not printed to avoid illegal modification by hand. */ - ltfsmsg(LTFS_ERR, 17292E); + ltfsmsg(LTFS_ERR, 172923); ret = -LTFS_LABEL_MISMATCH; } } diff --git a/src/libltfs/ltfs_fsops_raw.c b/src/libltfs/ltfs_fsops_raw.c index 2f6df457..8a2b6ca8 100644 --- a/src/libltfs/ltfs_fsops_raw.c +++ b/src/libltfs/ltfs_fsops_raw.c @@ -443,8 +443,9 @@ int ltfs_fsraw_add_extent(struct dentry *d, struct extent_info *ext, bool update int ltfs_fsraw_cleanup_extent(struct dentry *d, struct tc_position err_pos, unsigned long blocksize, struct ltfs_volume *vol) { int ret = 0; - struct name_list *entry, *tmp; + struct name_list *entry, *tmp; struct extent_info *ext, *preventry; + struct tc_position extent_last = {0, 0, UINT32_MAX, false, false}; if (HASH_COUNT(d->child_list) != 0) { HASH_ITER(hh, d->child_list, entry, tmp) { @@ -453,7 +454,24 @@ int ltfs_fsraw_cleanup_extent(struct dentry *d, struct tc_position err_pos, unsi } else { TAILQ_FOREACH_REVERSE_SAFE(ext, &entry->d->extentlist, extent_struct, list, preventry) { - if (err_pos.block <= (ext->start.block + ext->bytecount/blocksize)) { + if (ext->start.block && ext->bytecount) { + extent_last.partition = ltfs_part_id2num(ext->start.partition, vol); + /* Calculate the last block of this extent */ + extent_last.block = ext->start.block + (ext->bytecount / blocksize); + if ( (ext->bytecount % blocksize) == 0 ) + extent_last.block--; + } else { + extent_last.partition = UINT32_MAX; + extent_last.block = 0; + } + + /* + * err_pos has the first block number that tape drive has it on buffer + * but not transferred to the medium. + * It means position (err_pos-1) is the last block on the medium. + */ + if ( extent_last.partition == err_pos.partition && err_pos.block <= extent_last.block ) { + ltfsmsg(LTFS_INFO, 11334I, entry->name, (unsigned long long)ext->start.block, (unsigned long long)ext->bytecount); ret = ltfs_get_volume_lock(false, vol); diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index 4564d312..5c25c1fd 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -1122,30 +1122,32 @@ int tape_update_position(struct device_data *dev, struct tc_position *pos) return 0; } -int tape_get_physical_block_position(struct device_data *dev, struct tc_position *pos) +int tape_get_first_untransfered_position(struct device_data *dev, struct tc_position *pos) { int ret; - unsigned int block; CHECK_ARG_NULL(dev, -LTFS_NULL_ARG); CHECK_ARG_NULL(pos, -LTFS_NULL_ARG); + /* Update current position, just in case. Because no penalty here */ ret = dev->backend->readpos(dev->backend_data, &dev->position); if (ret < 0) { ltfsmsg(LTFS_ERR, 17132E); return ret; } - ret = dev->backend->get_block_in_buffer(dev->backend_data, &block); + /* Capture first untransferred position */ + ret = dev->backend->get_next_block_to_xfer(dev->backend_data, pos); if (ret < 0) { ltfsmsg(LTFS_ERR, 17132E); return ret; } - memcpy(pos, &dev->position, sizeof(struct tc_position)); - - ltfsmsg(LTFS_DEBUG, 11335D, (int)pos->block, block); - pos->block -= block; + ltfsmsg(LTFS_INFO, 17292I, + (unsigned long long)dev->position.partition, + (unsigned long long)dev->position.block, + (unsigned long long)pos->partition, + (unsigned long long)pos->block); return 0; } diff --git a/src/libltfs/tape.h b/src/libltfs/tape.h index 9bfaf108..ca340130 100644 --- a/src/libltfs/tape.h +++ b/src/libltfs/tape.h @@ -165,7 +165,7 @@ int tape_update_position(struct device_data *dev, struct tc_position *pos); int tape_seek(struct device_data *dev, struct tc_position *pos); int tape_seek_eod(struct device_data *dev, tape_partition_t partition); int tape_seek_append_position(struct device_data *dev, tape_partition_t prt, bool unlock_write); -int tape_get_physical_block_position(struct device_data *dev, struct tc_position *pos); +int tape_get_first_untransfered_position(struct device_data *dev, struct tc_position *pos); int tape_spacefm(struct device_data *dev, int count); ssize_t tape_read(struct device_data *dev, char *buf, size_t count, const bool unusual_size, diff --git a/src/libltfs/tape_ops.h b/src/libltfs/tape_ops.h index f4a51a8c..a2e3e967 100644 --- a/src/libltfs/tape_ops.h +++ b/src/libltfs/tape_ops.h @@ -915,12 +915,12 @@ struct tape_ops { int (*set_profiler)(void *device, char *work_dir, bool enable); /** - * Get block number stored in the drive buffer + * Get first block number which is not transferred to the medium yet in the buffer * @param device A pointer to the tape device - * @param block Number of blocks stored in the drive buffer + * @param pos Position of the record that is not transferred yet in the buffer * @return 0 on success or a negative value on error */ - int (*get_block_in_buffer)(void *device, unsigned int *block); + int (*get_next_block_to_xfer)(void *device, struct tc_position *pos); /** * Check if the generation of tape drive and the current loaded cartridge is read-only combination diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 5ae6ee9b..e1c5bc65 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -1196,10 +1196,10 @@ int camtape_unload(void *device, struct tc_position *pos) } /* - * Get the number of blocks in the buffer on the tape drive after a write. Eventually it would + * Get the first block position that is not transferred to the medium. Eventually it would * be nice to include this in status information returned from the sa(4) driver. */ -static int camtape_get_block_in_buffer(void *device, uint32_t *block) +static int camtape_get_next_block_to_xfer(void *device, struct tc_position *pos) { int rc; union ccb *ccb = NULL; @@ -1243,9 +1243,10 @@ static int camtape_get_block_in_buffer(void *device, uint32_t *block) if (rc != DEVICE_GOOD) camtape_process_errors(softc, rc, msg, "READPOS", true); else { - *block = scsi_3btoul(ext_data.num_objects); - ltfsmsg(LTFS_DEBUG, 30398D, "blocks-in-buffer", - (unsigned long long) *block, 0, 0, softc->drive_serial); + pos->partition = ext_data.partition; + pos->block = scsi_8btou64(ext_data.last_object) + ltfsmsg(LTFS_DEBUG, 30398D, "next-block-to-xfer", + (unsigned long long) pos->block, 0, 0, softc->drive_serial); } bailout: @@ -4176,7 +4177,7 @@ struct tape_ops camtape_drive_handler = { .get_serialnumber = camtape_get_serialnumber, .get_info = camtape_get_info, .set_profiler = camtape_set_profiler, - .get_block_in_buffer = camtape_get_block_in_buffer, + .get_next_block_to_xfer = camtape_get_next_block_to_xfer, .is_readonly = camtape_is_readonly }; diff --git a/src/tape_drivers/generic/file/filedebug_tc.c b/src/tape_drivers/generic/file/filedebug_tc.c index b9f6835b..74d8ca8c 100644 --- a/src/tape_drivers/generic/file/filedebug_tc.c +++ b/src/tape_drivers/generic/file/filedebug_tc.c @@ -814,7 +814,8 @@ int filedebug_write(void *device, const char *buf, size_t count, struct tc_posit return -EDEV_WRITE_PERM; } else if ( state->write_counter > (state->force_writeperm - THRESHOLD_FORCE_WRITE_NO_WRITE) ) { ltfsmsg(LTFS_INFO, 30019I); - pos->block++; + ++state->current_position.block; + pos->block = state->current_position.block; return DEVICE_GOOD; } } @@ -1872,8 +1873,7 @@ int filedebug_set_xattr(void *device, const char *name, const char *buf, size_t state->force_writeperm = perm_count; state->clear_by_pc = false; } - if (state->force_writeperm && state->force_writeperm < THRESHOLD_FORCE_WRITE_NO_WRITE) - state->force_writeperm = THRESHOLD_FORCE_WRITE_NO_WRITE; + state->write_counter = 0; ret = DEVICE_GOOD; } else if (! strcmp(name, "ltfs.vendor.IBM.forceErrorType")) { @@ -2776,9 +2776,13 @@ int filedebug_set_profiler(void *device, char *work_dir, bool enable) return 0; } -int filedebug_get_block_in_buffer(void *device, unsigned int *block) +int filedebug_get_next_block_to_xfer(void *device, struct tc_position *pos) { - *block = 0; + struct filedebug_data *state = (struct filedebug_data *)device; + + pos->partition = state->current_position.partition; + pos->block = state->current_position.block - THRESHOLD_FORCE_WRITE_NO_WRITE; + return 0; } @@ -2837,7 +2841,7 @@ struct tape_ops filedebug_handler = { .get_serialnumber = filedebug_get_serialnumber, .get_info = filedebug_get_info, .set_profiler = filedebug_set_profiler, - .get_block_in_buffer = filedebug_get_block_in_buffer, + .get_next_block_to_xfer = filedebug_get_next_block_to_xfer, .is_readonly = filedebug_is_readonly, }; diff --git a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c index ec00d949..133df2c0 100644 --- a/src/tape_drivers/generic/itdtimg/itdtimg_tc.c +++ b/src/tape_drivers/generic/itdtimg/itdtimg_tc.c @@ -1457,10 +1457,10 @@ int itdtimage_set_profiler(void *device, char *work_dir, bool enable) return 0; } -int itdtimage_get_block_in_buffer(void *device, unsigned int *block) +int itdtimage_get_next_block_to_xfer(void *device, struct tc_position *pos) { - *block = 0; - return 0; + /* This backend never accept write command */ + return -EDEV_WRITE_PROTECTED;; } /* Local functions */ @@ -1664,7 +1664,7 @@ struct tape_ops itdtimage_handler = { .get_serialnumber = itdtimage_get_serialnumber, .get_info = itdtimage_get_info, .set_profiler = itdtimage_set_profiler, - .get_block_in_buffer = itdtimage_get_block_in_buffer, + .get_next_block_to_xfer = itdtimage_get_next_block_to_xfer, .is_readonly = itdtimage_is_readonly, }; diff --git a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c index 146d8dc1..f8c1448c 100644 --- a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c +++ b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c @@ -2067,7 +2067,7 @@ int lin_tape_ibmtape_unload(void *device, struct tc_position *pos) } } -int lin_tape_ibmtape_get_block_in_buffer(void *device, uint32_t *block) +int lin_tape_ibmtape_get_next_block_to_xfer(void *device, struct tc_position *pos) { int rc; char *msg; @@ -2077,12 +2077,13 @@ int lin_tape_ibmtape_get_block_in_buffer(void *device, uint32_t *block) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_READPOS)); memset(&rp, 0, sizeof(struct read_tape_position)); - rp.data_format = RP_SHORT_FORM; + rp.data_format = RP_EXTENDED_FORM; - rc = _sioc_stioc_command(device, STIOC_READ_POSITION_EX, "READPOS SHORT", &rp, &msg); + rc = _sioc_stioc_command(device, STIOC_READ_POSITION_EX, "READPOS EXT", &rp, &msg); if (rc == DEVICE_GOOD) { - *block = (uint32_t)ltfs_betou32(rp.rp_data.rp_short.num_buffer_logical_obj); + pos->partition = rp.rp_data.rp_extended.active_partition; + pos->block = ltfs_betou64(rp.rp_data.rp_extended.last_logical_obj_position); } else { lin_tape_ibmtape_process_errors(device, rc, msg, "get block in buf", true); @@ -4419,7 +4420,7 @@ struct tape_ops lin_tape_ibmtape_drive_handler = { .get_serialnumber = lin_tape_ibmtape_get_serialnumber, .get_info = lin_tape_ibmtape_get_info, .set_profiler = lin_tape_ibmtape_set_profiler, - .get_block_in_buffer = lin_tape_ibmtape_get_block_in_buffer, + .get_next_block_to_xfer = lin_tape_ibmtape_get_next_block_to_xfer, .is_readonly = lin_tape_ibmtape_is_readonly, }; diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index 2b950d08..46517dcc 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -2755,7 +2755,7 @@ int sg_readpos(void *device, struct tc_position *pos) struct sg_data *priv = (struct sg_data*)device; sg_io_hdr_t req; - unsigned char cdb[CDB6_LEN]; + unsigned char cdb[CDB10_LEN]; unsigned char sense[MAXSENSE]; int timeout; char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "READPOS"; @@ -4965,14 +4965,14 @@ int sg_set_profiler(void *device, char *work_dir, bool enable) return rc; } -int sg_get_block_in_buffer(void *device, uint32_t *block) +int sg_get_next_block_to_xfer(void *device, struct tc_position *pos) { int ret = -EDEV_UNKNOWN; int ret_ep = DEVICE_GOOD; struct sg_data *priv = (struct sg_data*)device; sg_io_hdr_t req; - unsigned char cdb[CDB6_LEN]; + unsigned char cdb[CDB10_LEN]; unsigned char sense[MAXSENSE]; int timeout; char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "READPOS"; @@ -4981,6 +4981,8 @@ int sg_get_block_in_buffer(void *device, uint32_t *block) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_READPOS)); + memset(pos, 0, sizeof(struct tc_position)); + /* Zero out the CDB and the result buffer */ ret = init_sg_io_header(&req); if (ret < 0) @@ -4992,6 +4994,7 @@ int sg_get_block_in_buffer(void *device, uint32_t *block) /* Build CDB */ cdb[0] = READ_POSITION; cdb[1] = 0x08; /* Extended Format */ + ltfs_u16tobe(cdb + 7, sizeof(buf)); /* allocation length */ timeout = get_timeout(priv->timeouts, cdb[0]); if (timeout < 0) @@ -5010,10 +5013,11 @@ int sg_get_block_in_buffer(void *device, uint32_t *block) ret = sg_issue_cdb_command(&priv->dev, &req, &msg); if (ret == DEVICE_GOOD) { - *block = (buf[5] << 16) + (buf[6] << 8) + (int)buf[7]; + pos->partition = (tape_partition_t)buf[1]; + pos->block = ltfs_betou64(buf + 16); - ltfsmsg(LTFS_DEBUG, 30398D, "blocks-in-buffer", - (unsigned long long)*block, (unsigned long long)0, (unsigned long long)0, priv->drive_serial); + ltfsmsg(LTFS_DEBUG, 30398D, "next-block-to-xfer", + (unsigned long long)pos->partition, (unsigned long long)pos->block, (unsigned long long)0, priv->drive_serial); } else { ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); if (ret_ep < 0) @@ -5210,7 +5214,7 @@ struct tape_ops sg_handler = { .get_serialnumber = sg_get_serialnumber, .get_info = sg_get_info, .set_profiler = sg_set_profiler, - .get_block_in_buffer = sg_get_block_in_buffer, + .get_next_block_to_xfer = sg_get_next_block_to_xfer, .is_readonly = sg_is_readonly, }; diff --git a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c index 38e50173..0ea8df55 100644 --- a/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c +++ b/src/tape_drivers/netbsd/scsipi-ibmtape/scsipi_ibmtape.c @@ -2377,7 +2377,7 @@ int scsipi_ibmtape_readpos(void *device, struct tc_position *pos) struct scsipi_ibmtape_data *priv = (struct scsipi_ibmtape_data*)device; scsireq_t req; - unsigned char cdb[CDB6_LEN]; + unsigned char cdb[CDB10_LEN]; int timeout; char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "READPOS"; char *msg = NULL; @@ -4505,14 +4505,14 @@ int scsipi_ibmtape_set_profiler(void *device, char *work_dir, bool enable) return rc; } -int scsipi_ibmtape_get_block_in_buffer(void *device, uint32_t *block) +int scsipi_ibmtape_get_next_block_to_xfer(void *device, struct tc_position *pos) { int ret = -EDEV_UNKNOWN; int ret_ep = DEVICE_GOOD; struct scsipi_ibmtape_data *priv = (struct scsipi_ibmtape_data*)device; scsireq_t req; - unsigned char cdb[CDB6_LEN]; + unsigned char cdb[CDB10_LEN]; int timeout; char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "READPOS"; char *msg = NULL; @@ -4520,6 +4520,8 @@ int scsipi_ibmtape_get_block_in_buffer(void *device, uint32_t *block) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_READPOS)); + memset(pos, 0, sizeof(struct tc_position)); + /* Zero out the CDB and the result buffer */ ret = init_scsireq(&req); if (ret < 0) @@ -4530,6 +4532,7 @@ int scsipi_ibmtape_get_block_in_buffer(void *device, uint32_t *block) /* Build CDB */ cdb[0] = READ_POSITION; cdb[1] = 0x08; /* Extended Format */ + ltfs_u16tobe(cdb + 7, sizeof(buf)); /* allocation length */ timeout = ibm_tape_get_timeout(priv->timeouts, cdb[0]); if (timeout < 0) @@ -4545,10 +4548,11 @@ int scsipi_ibmtape_get_block_in_buffer(void *device, uint32_t *block) ret = scsipi_issue_cdb_command(&priv->dev, &req, cmd_desc, &msg); if (ret == DEVICE_GOOD) { - *block = (buf[5] << 16) + (buf[6] << 8) + (int)buf[7]; + pos->partition = (tape_partition_t)buf[1]; + pos->block = ltfs_betou64(buf + 16); - ltfsmsg(LTFS_DEBUG, 30398D, "blocks-in-buffer", - (unsigned long long)*block, (unsigned long long)0, (unsigned long long)0, priv->drive_serial); + ltfsmsg(LTFS_DEBUG, 30398D, "next-block-to-xfer", + (unsigned long long)pos->partition, (unsigned long long)pos->block, (unsigned long long)0, priv->drive_serial); } else { ret_ep = _process_errors(device, ret, msg, cmd_desc, true, true); if (ret_ep < 0) @@ -4616,7 +4620,7 @@ struct tape_ops scsipi_ibmtape_handler = { .get_serialnumber = scsipi_ibmtape_get_serialnumber, .get_info = scsipi_ibmtape_get_info, .set_profiler = scsipi_ibmtape_set_profiler, - .get_block_in_buffer = scsipi_ibmtape_get_block_in_buffer, + .get_next_block_to_xfer = scsipi_ibmtape_get_next_block_to_xfer, .is_readonly = scsipi_ibmtape_is_readonly, }; diff --git a/src/tape_drivers/osx/iokit/iokit_tape.c b/src/tape_drivers/osx/iokit/iokit_tape.c index 91d2c0b6..680615ab 100644 --- a/src/tape_drivers/osx/iokit/iokit_tape.c +++ b/src/tape_drivers/osx/iokit/iokit_tape.c @@ -2126,7 +2126,7 @@ int iokit_readpos(void *device, struct tc_position *pos) struct iokit_data *priv = (struct iokit_data*)device; struct iokit_scsi_request req; - unsigned char cdb[CDB6_LEN]; + unsigned char cdb[CDB10_LEN]; int timeout; char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "READPOS"; char *msg = NULL; @@ -4238,13 +4238,13 @@ int iokit_set_profiler(void *device, char *work_dir, bool enable) return rc; } -int iokit_get_block_in_buffer(void *device, uint32_t *block) +int iokit_get_next_block_to_xfer(void *device, struct tc_position *pos) { int ret = -EDEV_UNKNOWN; struct iokit_data *priv = (struct iokit_data*)device; struct iokit_scsi_request req; - unsigned char cdb[CDB6_LEN]; + unsigned char cdb[CDB10_LEN]; int timeout; char cmd_desc[COMMAND_DESCRIPTION_LENGTH] = "READPOS"; char *msg = NULL; @@ -4252,12 +4252,16 @@ int iokit_get_block_in_buffer(void *device, uint32_t *block) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_READPOS)); + memset(pos, 0, sizeof(struct tc_position)); + /* Zero out the CDB and the result buffer */ memset(cdb, 0, sizeof(cdb)); + memset(&req, 0, sizeof(struct iokit_scsi_request)); /* Build CDB */ cdb[0] = READ_POSITION; cdb[1] = 0x08; /* Extended Format */ + ltfs_u16tobe(cdb + 7, sizeof(buf)); /* allocation length */ timeout = get_timeout(priv->timeouts, cdb[0]); if (timeout < 0) @@ -4276,10 +4280,11 @@ int iokit_get_block_in_buffer(void *device, uint32_t *block) ret = iokit_issue_cdb_command(&priv->dev, &req, &msg); if (ret == DEVICE_GOOD) { - *block = (buf[5] << 16) + (buf[6] << 8) + (int)buf[7]; + pos->partition = (tape_partition_t)buf[1]; + pos->block = ltfs_betou64(buf + 16); - ltfsmsg(LTFS_DEBUG, 30998D, "blocks-in-buffer", - (unsigned long long) *block, (unsigned long long)0, (unsigned long long)0, priv->drive_serial); + ltfsmsg(LTFS_DEBUG, 30998D, "next-block-to-xfer", + (unsigned long long)pos->partition, (unsigned long long)pos->block, (unsigned long long)0, priv->drive_serial); } else { _process_errors(device, ret, msg, cmd_desc, true); } @@ -4345,7 +4350,7 @@ struct tape_ops iokit_handler = { .get_serialnumber = iokit_get_serialnumber, .get_info = iokit_get_info, .set_profiler = iokit_set_profiler, - .get_block_in_buffer = iokit_get_block_in_buffer, + .get_next_block_to_xfer = iokit_get_next_block_to_xfer, .is_readonly = iokit_is_readonly, }; diff --git a/src/tape_drivers/tape_drivers.h b/src/tape_drivers/tape_drivers.h index 6ca2ce93..fa797a18 100644 --- a/src/tape_drivers/tape_drivers.h +++ b/src/tape_drivers/tape_drivers.h @@ -89,7 +89,8 @@ typedef int (*crc_check)(void *buf, size_t n); typedef void* (*memcpy_crc_enc)(void *dest, const void *src, size_t n); typedef int (*memcpy_crc_check)(void *dest, const void *src, size_t n); -#define THRESHOLD_FORCE_WRITE_NO_WRITE (5) + +#define THRESHOLD_FORCE_WRITE_NO_WRITE (20) #define DEFAULT_WRITEPERM (0) #define DEFAULT_READPERM (0) #define DEFAULT_ERRORTYPE (0) From 044a0e3a93750d42af7361f827cb3c581a979a88 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 10 Aug 2022 10:42:36 +0900 Subject: [PATCH 087/121] Correct a message ID --- src/libltfs/ltfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 87a2cdf2..5dcb51cf 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1959,7 +1959,7 @@ int ltfs_mount_indexfile(char* filename, bool label_check, struct ltfs_volume *v * Volume UUID in label and it on index is not matched. * Actual UUID is not printed to avoid illegal modification by hand. */ - ltfsmsg(LTFS_ERR, 172923); + ltfsmsg(LTFS_ERR, 17292E); ret = -LTFS_LABEL_MISMATCH; } } From 8a0f1169ec7713c2b498c978e621c9c379c6508a Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 10 Aug 2022 10:47:47 +0900 Subject: [PATCH 088/121] Correct a message ID --- src/libltfs/ltfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 5dcb51cf..c085c2a5 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1959,7 +1959,7 @@ int ltfs_mount_indexfile(char* filename, bool label_check, struct ltfs_volume *v * Volume UUID in label and it on index is not matched. * Actual UUID is not printed to avoid illegal modification by hand. */ - ltfsmsg(LTFS_ERR, 17292E); + ltfsmsg(LTFS_ERR, 17293E); ret = -LTFS_LABEL_MISMATCH; } } From bf58bc31ae50dfee12cd4c96cc59a5aacc415bb2 Mon Sep 17 00:00:00 2001 From: Chris Dinneen Date: Mon, 3 Oct 2022 16:01:21 +0800 Subject: [PATCH 089/121] Cleanup language on READMEmd. (#362) --- README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 208329ba..4b504d57 100644 --- a/README.md +++ b/README.md @@ -53,13 +53,13 @@ The table below show status of the LTFS format Specification # How to use the LTFS (Quick start) -This section is for person who already have a machine the LTFS is installed. Instruction how to use the LTFS is also available on [Wiki](https://github.com/LinearTapeFileSystem/ltfs/wiki). +This section is for a person who already has a machine with the LTFS installed. Instructions on how to use the LTFS is also available on [Wiki](https://github.com/LinearTapeFileSystem/ltfs/wiki). ## Step1: List tape drives `# ltfs -o device_list` -The output is like follows. You can have 3 drives in this example and you can use "Device Name" field, like `/dev/sg43` in this case, as the argument of ltfs command to mount the tape drive. +The output is as follows. You have 3 drives in this example and you can use "Device Name" field, like `/dev/sg43` in this case, as the argument of ltfs command to mount the tape drive. ``` 50c4 LTFS14000I LTFS starting, LTFS version 2.4.0.0 (10022), log level 2. @@ -80,13 +80,13 @@ Device Name = /dev/sg37, Vender ID = IBM , Product ID = ULT3580-TD7 , Seri ## Step2: Format a tape -As described into the LTFS format specifications, LTFS uses the partition feature of the tape drive. It means you can't use a tape just after you purchase a tape. You need format the tape before using int on LTFS. +As described in the LTFS format specifications, LTFS uses the partition feature of the tape drive. This means you can't use a tape just after you purchase a tape. You need format the tape before using it on LTFS. To format a tape, you can use `mkltfs` command like `# mkltfs -d 9A700L0077` -In this case, `mkltfs` tries to format a tape in the tape drive `9A700L0077`. You can use a device name `/dev/sg43` instead. +In this case, `mkltfs` tries to format a tape in the tape drive `9A700L0077`. You can use the device name `/dev/sg43` instead. ## Step3: Mount a tape through a tape drive @@ -94,29 +94,29 @@ After you prepared a formatted tape, you can mount it through a tape drive like `# ltfs -o devname=9A700L0077 /ltfs` -In this command, the ltfs command try to mount the tape in the tape drive `9A700L0077` to `/ltfs` directory. Of cause, you can use a device name `/dev/sg43` instead. +In this command, the ltfs command will try to mount the tape in the tape drive `9A700L0077` to `/ltfs` directory. Of course, you can use a device name `/dev/sg43` instead. -If mount process is successfully done, you can access to the LTFS tape through `/ltfs` directory. +If the mount process is successfully done, you can access to the LTFS tape through `/ltfs` directory. You must not touch any `st` devices while ltfs is mounting a tape. ## Step4: Unmount the tape drive -You can use following command when you want to unmount the tape. The ltfs command try to write down the current meta-data to the tape and close the tape cleanly. +You can use following command when you want to unmount the tape. The ltfs command try to write the current meta-data to the tape and close the tape cleanly. `# umount /ltfs` -One thing you need to pay attention here is it is not a unmount completion when umount command is returned. It just a finish of trigger to notify the unmount request to the ltfs command. Actual unmount is completed when the ltfs command is finished. +One thing you need to pay attention to here is, that the unmount command continues to work in the background after it returns. It just initiates a trigger to notify the the ltfs command of the unmount request. Actual unmount is completed when the ltfs command is finished. ## The `ltfsee_ordered_copy` utility The [`ltfsee_ordered_copy`](https://github.com/LinearTapeFileSystem/ltfs/wiki/ltfs_ordered_copy) is a program to copy files from source to destination with LTFS order optimization. -It is written by python and it can work on both python2 and python3 (Python 2.7 or later is strongly recommended). You need to install the `pyxattr` module for both python2 and python3. +It is written in python and it can work with both python2 and python3 (Python 2.7 or later is strongly recommended). You need to install the `pyxattr` module for both python2 and python3. # Building the LTFS from this GitHub project -These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. +These instructions will get a copy of the project up and running on your local machine for development and testing purposes. ## Prerequisites for build From 2d7d64230795aab59229a2fb3c2dd1b87c77de13 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 16 Nov 2022 16:21:58 +0900 Subject: [PATCH 090/121] Modify linker flags (#366) Current linker flag for creating shared libraries are not good. The linker called from libtool reports errors of unexisted files. This modifications fixes linking problem from 3rd party apps which links the shared libraries provided from LTFS. --- src/Makefile.am | 2 +- src/iosched/Makefile.am | 4 ++-- src/kmi/Makefile.am | 4 ++-- src/libltfs/Makefile.am | 2 +- src/tape_drivers/freebsd/cam/Makefile.am | 6 +++--- src/tape_drivers/generic/file/Makefile.am | 2 +- src/tape_drivers/generic/itdtimg/Makefile.am | 2 +- src/tape_drivers/linux/lin_tape/Makefile.am | 2 +- src/tape_drivers/linux/sg/Makefile.am | 2 +- src/tape_drivers/netbsd/scsipi-ibmtape/Makefile.am | 2 +- src/tape_drivers/osx/iokit/Makefile.am | 2 +- src/utils/Makefile.am | 4 ++-- 12 files changed, 17 insertions(+), 17 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 8dd6fce0..230239f6 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -40,7 +40,7 @@ ltfs_SOURCES = main.c ltfs_fuse.c ltfs_DEPENDENCIES = libltfs/libltfs.la ../messages/libbin_ltfs_dat.a ltfs_CPPFLAGS = @AM_CPPFLAGS@ -I ../ltfs-sde/src -fPIC ltfs_LDADD = libltfs/libltfs.la -ltfs_LDFLAGS = @AM_LDFLAGS@ -L../messages -lbin_ltfs_dat +ltfs_LDFLAGS = @AM_LDFLAGS@ ../messages/libbin_ltfs_dat.a endif PLAT_DRV = diff --git a/src/iosched/Makefile.am b/src/iosched/Makefile.am index 698a9505..d3887662 100644 --- a/src/iosched/Makefile.am +++ b/src/iosched/Makefile.am @@ -39,13 +39,13 @@ BASENAMES = libiosched-fcfs libiosched-unified AM_LIBTOOLFLAGS = --tag=disable-static libiosched_fcfs_la_SOURCES = fcfs.c -libiosched_fcfs_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../messages/ -liosched_fcfs_dat +libiosched_fcfs_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../messages/libiosched_fcfs_dat.a libiosched_fcfs_la_DEPENDENCIES = ../../messages/libiosched_fcfs_dat.a ../libltfs/libltfs.la libiosched_fcfs_la_LIBADD = ../libltfs/libltfs.la libiosched_fcfs_la_CPPFLAGS = @AM_CPPFLAGS@ -I .. libiosched_unified_la_SOURCES = unified.c cache_manager.c -libiosched_unified_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../messages/ -liosched_unified_dat +libiosched_unified_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../messages/libiosched_unified_dat.a libiosched_unified_la_DEPENDENCIES = ../../messages/libiosched_unified_dat.a ../libltfs/libltfs.la libiosched_unified_la_LIBADD = ../libltfs/libltfs.la libiosched_unified_la_CPPFLAGS = @AM_CPPFLAGS@ -I .. diff --git a/src/kmi/Makefile.am b/src/kmi/Makefile.am index 726a3aae..7aa05d94 100644 --- a/src/kmi/Makefile.am +++ b/src/kmi/Makefile.am @@ -41,13 +41,13 @@ AM_LIBTOOLFLAGS = --tag=disable-static libkmi_simple_la_SOURCES = simple.c key_format_ltfs.c libkmi_simple_la_DEPENDENCIES = ../../messages/libkmi_simple_dat.a ../libltfs/libltfs.la libkmi_simple_la_LIBADD = ../libltfs/libltfs.la -libkmi_simple_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../messages -lkmi_simple_dat +libkmi_simple_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../messages/libkmi_simple_dat.a libkmi_simple_la_CPPFLAGS = @AM_CPPFLAGS@ -I .. -DKMI_SIMPLE libkmi_flatfile_la_SOURCES = flatfile.c key_format_ltfs.c libkmi_flatfile_la_DEPENDENCIES = ../../messages/libkmi_flatfile_dat.a ../libltfs/libltfs.la libkmi_flatfile_la_LIBADD = ../libltfs/libltfs.la -libkmi_flatfile_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../messages -lkmi_flatfile_dat +libkmi_flatfile_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../messages/libkmi_flatfile_dat.a libkmi_flatfile_la_CPPFLAGS = @AM_CPPFLAGS@ -I .. install-exec-hook: diff --git a/src/libltfs/Makefile.am b/src/libltfs/Makefile.am index 22f22fcb..b3ae064d 100644 --- a/src/libltfs/Makefile.am +++ b/src/libltfs/Makefile.am @@ -71,7 +71,7 @@ libltfs_la_SOURCES = \ libltfs_la_DEPENDENCIES = ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a libltfs_la_LIBADD = libltfs_la_CPPFLAGS = @AM_CPPFLAGS@ @AM_EXTRA_CPPFLAGS@ @AM_EXTRA_CPPFLAGS@ -I .. -libltfs_la_LDFLAGS = @AM_LDFLAGS@ -L../../messages -llibltfs_dat -linternal_error_dat -ltape_common_dat +libltfs_la_LDFLAGS = @AM_LDFLAGS@ ../../messages/liblibltfs_dat.a ../../messages/libinternal_error_dat.a ../../messages/libtape_common_dat.a install-data-local: if [ ! -d "$(DESTDIR)$(prefix)/share/snmp" ]; then \ diff --git a/src/tape_drivers/freebsd/cam/Makefile.am b/src/tape_drivers/freebsd/cam/Makefile.am index 6127f989..5498be6c 100644 --- a/src/tape_drivers/freebsd/cam/Makefile.am +++ b/src/tape_drivers/freebsd/cam/Makefile.am @@ -41,9 +41,9 @@ BASENAMES = libtape-cam AM_LIBTOOLFLAGS = --tag=disable-static libtape_cam_la_SOURCES = cam_cmn.c cam_tc.c vendor_compat.c ibm_tape.c hp_tape.c quantum_tape.c -libtape_cam_la_DEPENDENCIES = ../../../../messages/libtape_freebsd_cam_dat.a ../../../libltfs/libltfs.la libtape_cam_la-reed_solomon_crc.lo libtape_cam_la-crc32c_crc.lo -libtape_cam_la_LIBADD = ../../../libltfs/libltfs.la ./libtape_cam_la-reed_solomon_crc.lo ./libtape_cam_la-crc32c_crc.lo -libtape_cam_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../../../messages -ltape_freebsd_cam_dat +libtape_cam_la_DEPENDENCIES = ../../../../messages/libtape_freebsd_cam_dat.a ../../../libltfs/libltfs.la libtape_cam_la-reed_solomon_crc.lo libtape_cam_la-crc32c_crc.lo +libtape_cam_la_LIBADD = ../../../libltfs/libltfs.la ./libtape_cam_la-reed_solomon_crc.lo ./libtape_cam_la-crc32c_crc.lo +libtape_cam_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../../../messages/libtape_freebsd_cam_dat.a libtape_cam_la_CPPFLAGS = @AM_CPPFLAGS@ -I ../../.. -I ../.. vendor_compat.c: diff --git a/src/tape_drivers/generic/file/Makefile.am b/src/tape_drivers/generic/file/Makefile.am index 264fcd0e..661043d5 100644 --- a/src/tape_drivers/generic/file/Makefile.am +++ b/src/tape_drivers/generic/file/Makefile.am @@ -41,7 +41,7 @@ AM_LIBTOOLFLAGS = --tag=disable-static libtape_file_la_SOURCES = filedebug_tc.c filedebug_conf_tc.c ibm_tape.c libtape_file_la_DEPENDENCIES = ../../../../messages/libtape_generic_file_dat.a ../../../libltfs/libltfs.la libtape_file_la_LIBADD =../../../libltfs/libltfs.la -libtape_file_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../../../messages/ -ltape_generic_file_dat +libtape_file_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../../../messages/libtape_generic_file_dat.a libtape_file_la_CPPFLAGS = @AM_CPPFLAGS@ -I ../../.. ibm_tape.c: diff --git a/src/tape_drivers/generic/itdtimg/Makefile.am b/src/tape_drivers/generic/itdtimg/Makefile.am index 1504efb0..655eeb37 100644 --- a/src/tape_drivers/generic/itdtimg/Makefile.am +++ b/src/tape_drivers/generic/itdtimg/Makefile.am @@ -41,7 +41,7 @@ AM_LIBTOOLFLAGS = --tag=disable-static libtape_itdtimg_la_SOURCES = itdtimg_tc.c libtape_itdtimg_la_DEPENDENCIES = ../../../../messages/libtape_generic_itdtimg_dat.a ../../../libltfs/libltfs.la libtape_itdtimg_la_LIBADD = ../../../libltfs/libltfs.la -libtape_itdtimg_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../../../messages -ltape_generic_itdtimg_dat +libtape_itdtimg_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../../../messages/libtape_generic_itdtimg_dat.a libtape_itdtimg_la_CPPFLAGS = @AM_CPPFLAGS@ -I ../../.. install-exec-hook: diff --git a/src/tape_drivers/linux/lin_tape/Makefile.am b/src/tape_drivers/linux/lin_tape/Makefile.am index 608270b0..57335706 100644 --- a/src/tape_drivers/linux/lin_tape/Makefile.am +++ b/src/tape_drivers/linux/lin_tape/Makefile.am @@ -41,7 +41,7 @@ AM_LIBTOOLFLAGS = --tag=disable-static libtape_lin_tape_la_SOURCES = lin_tape_ibmtape.c vendor_compat.c ibm_tape.c hp_tape.c quantum_tape.c libtape_lin_tape_la_DEPENDENCIES = ../../../../messages/libtape_linux_lin_tape_dat.a ../../../libltfs/libltfs.la ./libtape_lin_tape_la-reed_solomon_crc.lo ./libtape_lin_tape_la-crc32c_crc.lo libtape_lin_tape_la_LIBADD = ../../../libltfs/libltfs.la ./libtape_lin_tape_la-reed_solomon_crc.lo ./libtape_lin_tape_la-crc32c_crc.lo -libtape_lin_tape_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../../../messages -ltape_linux_lin_tape_dat +libtape_lin_tape_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../../../messages/libtape_linux_lin_tape_dat.a libtape_lin_tape_la_CPPFLAGS = @AM_CPPFLAGS@ -I ../../.. -I ../.. vendor_compat.c: diff --git a/src/tape_drivers/linux/sg/Makefile.am b/src/tape_drivers/linux/sg/Makefile.am index 1e5948e4..6cc2fa07 100644 --- a/src/tape_drivers/linux/sg/Makefile.am +++ b/src/tape_drivers/linux/sg/Makefile.am @@ -41,7 +41,7 @@ AM_LIBTOOLFLAGS = --tag=disable-static libtape_sg_la_SOURCES = sg_scsi_tape.c sg_tape.c vendor_compat.c ibm_tape.c hp_tape.c quantum_tape.c open_factor.c libtape_sg_la_DEPENDENCIES = ../../../../messages/libtape_linux_sg_dat.a ../../../libltfs/libltfs.la ./libtape_sg_la-reed_solomon_crc.lo ./libtape_sg_la-crc32c_crc.lo libtape_sg_la_LIBADD = ../../../libltfs/libltfs.la ./libtape_sg_la-reed_solomon_crc.lo ./libtape_sg_la-crc32c_crc.lo -libtape_sg_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../../../messages -ltape_linux_sg_dat +libtape_sg_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../../../messages/libtape_linux_sg_dat.a libtape_sg_la_CPPFLAGS = @AM_CPPFLAGS@ @AM_EXTRA_CPPFLAGS@ -I ../../.. -I ../.. vendor_compat.c: diff --git a/src/tape_drivers/netbsd/scsipi-ibmtape/Makefile.am b/src/tape_drivers/netbsd/scsipi-ibmtape/Makefile.am index 64ac236a..1515a5ed 100644 --- a/src/tape_drivers/netbsd/scsipi-ibmtape/Makefile.am +++ b/src/tape_drivers/netbsd/scsipi-ibmtape/Makefile.am @@ -41,7 +41,7 @@ AM_LIBTOOLFLAGS = --tag=disable-static libtape_scsipi_ibmtape_la_SOURCES = scsipi_scsi_tape.c scsipi_ibmtape.c vendor_compat.c ibm_tape.c hp_tape.c quantum_tape.c libtape_scsipi_ibmtape_la_DEPENDENCIES = ../../../../messages/libtape_linux_sg_ibmtape_dat.a ../../../libltfs/libltfs.la libtape_scsipi_ibmtape_la-reed_solomon_crc.lo libtape_scsipi_ibmtape_la-crc32c_crc.lo libtape_scsipi_ibmtape_la_LIBADD = ../../../libltfs/libltfs.la libtape_scsipi_ibmtape_la-reed_solomon_crc.lo libtape_scsipi_ibmtape_la-crc32c_crc.lo -libtape_scsipi_ibmtape_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../../../messages -ltape_linux_sg_ibmtape_dat +libtape_scsipi_ibmtape_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../../../messages/libtape_linux_sg_ibmtape_dat.a libtape_scsipi_ibmtape_la_CPPFLAGS = @AM_CPPFLAGS@ @AM_EXTRA_CPPFLAGS@ -I ../../.. -I ../.. vendor_compat.c: diff --git a/src/tape_drivers/osx/iokit/Makefile.am b/src/tape_drivers/osx/iokit/Makefile.am index 172f00d6..87d80818 100644 --- a/src/tape_drivers/osx/iokit/Makefile.am +++ b/src/tape_drivers/osx/iokit/Makefile.am @@ -41,7 +41,7 @@ AM_LIBTOOLFLAGS = --tag=disable-static libtape_iokit_la_SOURCES = iokit_tape.c iokit_scsi.c iokit_service.c vendor_compat.c ibm_tape.c hp_tape.c quantum_tape.c libtape_iokit_la_DEPENDENCIES = ../../../../messages/libtape_iokit_dat.a ../../../libltfs/libltfs.la ./libtape_iokit_la-reed_solomon_crc.lo ./libtape_iokit_la-crc32c_crc.lo libtape_iokit_la_LIBADD = ../../../libltfs/libltfs.la ./libtape_iokit_la-reed_solomon_crc.lo ./libtape_iokit_la-crc32c_crc.lo -libtape_iokit_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ -L../../../../messages -ltape_iokit_dat +libtape_iokit_la_LDFLAGS = -avoid-version -module @AM_LDFLAGS@ ../../../../messages/libtape_iokit_dat.a libtape_iokit_la_CPPFLAGS = @AM_CPPFLAGS@ @AM_EXTRA_CPPFLAGS@ -I ../../.. -I ../.. vendor_compat.c: diff --git a/src/utils/Makefile.am b/src/utils/Makefile.am index 1a7a7e3c..918d8365 100644 --- a/src/utils/Makefile.am +++ b/src/utils/Makefile.am @@ -42,13 +42,13 @@ noinst_HEADERS = mkltfs_SOURCES = mkltfs.c mkltfs_DEPENDENCIES = ../libltfs/libltfs.la ../../messages/libbin_mkltfs_dat.a mkltfs_LDADD = ../libltfs/libltfs.la -mkltfs_LDFLAGS = @AM_LDFLAGS@ -L../../messages -lbin_mkltfs_dat +mkltfs_LDFLAGS = @AM_LDFLAGS@ ../../messages/libbin_mkltfs_dat.a mkltfs_CPPFLAGS = @AM_CPPFLAGS@ -I .. -fPIC ltfsck_SOURCES = ltfsck.c ltfsck_DEPENDENCIES = ../libltfs/libltfs.la ../../messages/libbin_ltfsck_dat.a ltfsck_LDADD = ../libltfs/libltfs.la -ltfsck_LDFLAGS = @AM_LDFLAGS@ -L../../messages -lbin_ltfsck_dat +ltfsck_LDFLAGS = @AM_LDFLAGS@ ../../messages/libbin_ltfsck_dat.a ltfsck_CPPFLAGS = @AM_CPPFLAGS@ -I .. -fPIC ltfsindextool_SOURCES = ltfsindextool.c From 56572c520ee64cc4ba902e52adf91047158df15d Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Sat, 19 Nov 2022 13:29:24 +0900 Subject: [PATCH 091/121] Install config_file.h as a header (#368) Add config_file.h into headers that shall be installed. --- .github/workflows/codeql-analysis.yml | 24 ++++++++++++------------ src/Makefile.am | 2 +- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml index 4dec8b88..d7aec110 100644 --- a/.github/workflows/codeql-analysis.yml +++ b/.github/workflows/codeql-analysis.yml @@ -43,7 +43,7 @@ jobs: # Initializes the CodeQL tools for scanning. - name: Initialize CodeQL - uses: github/codeql-action/init@v1 + uses: github/codeql-action/init@v2 with: languages: ${{ matrix.language }} # If you wish to specify custom queries, you can do so here or in a config file. @@ -53,8 +53,8 @@ jobs: # Autobuild attempts to build any compiled languages (C/C++, C#, or Java). # If this step fails, then you should remove it and run the build manually (see below) - - name: Autobuild - uses: github/codeql-action/autobuild@v1 + #- name: Autobuild + # uses: github/codeql-action/autobuild@v2 # ℹ️ Command-line programs to run using the OS shell. # 📚 https://git.io/JvXDl @@ -63,14 +63,14 @@ jobs: # and modify them (or add more) to build your code if your project # uses a compiled language - #- run: | - # sudo apt-get -q -y update - # sudo apt-get -q -y upgrade - # sudo apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev - # sudo cp .github/workflows/icu-config /usr/bin/icu-config - # ./autogen.sh - # ./configure - # make + - run: | + sudo apt-get -q -y update + sudo apt-get -q -y upgrade + sudo apt-get -y install build-essential automake autoconf libtool pkg-config icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev + sudo cp .github/workflows/icu-config /usr/bin/icu-config + ./autogen.sh + ./configure + make - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@v1 + uses: github/codeql-action/analyze@v2 diff --git a/src/Makefile.am b/src/Makefile.am index 230239f6..4469ba8a 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -77,7 +77,6 @@ SUBDIRS = libltfs $(GEN_DRV) $(PLAT_DRV) $(SUB) iosched utils noinst_HEADERS = \ libltfs/base64.h \ - libltfs/config_file.h \ libltfs/dcache.h \ libltfs/fs.h \ libltfs/index_criteria.h \ @@ -100,6 +99,7 @@ noinst_HEADERS = \ libltfs/arch/version.h nobase_pkginclude_HEADERS = \ + libltfs/config_file.h \ libltfs/dcache_ops.h \ libltfs/iosched_ops.h \ libltfs/kmi_ops.h \ From d7230c8b3fef419955bfaafeec65d9fd16393a94 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 7 Dec 2022 20:32:16 +0900 Subject: [PATCH 092/121] Fix a type mismatch bug in ltfs_ordered_copy (#372) This change also includes a bug for single file copy bug. --- src/utils/ltfs_ordered_copy | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/utils/ltfs_ordered_copy b/src/utils/ltfs_ordered_copy index 900ec7d1..d9809909 100755 --- a/src/utils/ltfs_ordered_copy +++ b/src/utils/ltfs_ordered_copy @@ -334,6 +334,9 @@ if args.DEST == None: logger.error('No destination is specified') exit(2) +if args.keep_tree is None: + args.keep_tree = '' + # Special case: # Copy source is only one file if args.recursive == False and len(args.SOURCE) == 1: @@ -361,7 +364,7 @@ direct_write_threads = 8 try: sig = xattr.get(args.DEST, VEA_PREFIX + LTFS_SIG_VEA) - if sig.startswith("LTFS"): + if sig.startswith(b"LTFS"): logger.info("Destination {0} is LTFS".format(args.DEST)) direct_write_threads = 1 else: @@ -385,9 +388,6 @@ if len(args.SOURCE) == 0: args.SOURCE.append(line.rstrip('\r\n')) logger.log(NOTSET + 1, 'Source: {}'.format(args.SOURCE)) -if args.keep_tree is None: - args.keep_tree = '' - # Create the list of copy item copyq = CopyQueue(logger) for s in args.SOURCE: From d8d72a7954551bc4051576e15e76d7fed68266e6 Mon Sep 17 00:00:00 2001 From: liu shiwei Date: Wed, 7 Dec 2022 19:36:54 +0800 Subject: [PATCH 093/121] Fix gcc error message by unexpected extra options (#375) Fix the bug adding extra parameters " 2> /dev/null" to the gcc command line --- configure.ac | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index da2d0d04..ff5b017c 100644 --- a/configure.ac +++ b/configure.ac @@ -372,7 +372,7 @@ if test "x${snmp}" != "xno" then SNMP_ENABLE="-D ENABLE_SNMP" SNMP_MODULE_CFLAGS="`net-snmp-config --cflags 2> /dev/null`"; - SNMP_MODULE_LIBS_A="`net-snmp-config --agent-libs` 2> /dev/null"; + SNMP_MODULE_LIBS_A="`net-snmp-config --agent-libs 2> /dev/null`"; SNMP_MODULE_LIBS="`net-snmp-config --libs 2> /dev/null`"; if test -z "$SNMP_MODULE_LIBS_A" then From 2db929b5fc641fbeccb06ba38b3c7e3d4f25a86c Mon Sep 17 00:00:00 2001 From: liu shiwei Date: Thu, 8 Dec 2022 13:43:23 +0800 Subject: [PATCH 094/121] Replace specifying options for ICU from icu-config to pkg-config (#376) --- build.sh | 3 +-- configure.ac | 6 +++--- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/build.sh b/build.sh index 8875035a..2b967245 100755 --- a/build.sh +++ b/build.sh @@ -5,8 +5,7 @@ sudo apt-get -q -y update sudo apt-get -q -y upgrade -sudo apt-get -y install build-essential automake autoconf libtool pkg-config libicu66 icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev -sudo cp .github/workflows/icu-config /usr/bin/icu-config +sudo apt-get -y install build-essential automake autoconf libtool pkg-config icu-devtools libicu-dev libxml2-dev uuid-dev fuse libfuse-dev libsnmp-dev ./autogen.sh ./configure make diff --git a/configure.ac b/configure.ac index ff5b017c..b579b92f 100644 --- a/configure.ac +++ b/configure.ac @@ -329,9 +329,9 @@ dnl dnl Check for ICU dnl AC_MSG_CHECKING(ICU version) -ICU_MODULE_VERSION="`icu-config --version 2> /dev/null`"; -ICU_MODULE_CFLAGS=" `icu-config --cppflags 2> /dev/null`"; -ICU_MODULE_LIBS="`icu-config --ldflags 2> /dev/null`"; +ICU_MODULE_VERSION="`pkg-config --modversion icu-i18n 2> /dev/null`"; +ICU_MODULE_CFLAGS="`pkg-config --cflags icu-i18n 2> /dev/null`"; +ICU_MODULE_LIBS="`pkg-config --libs icu-i18n 2> /dev/null`"; AC_MSG_RESULT([$ICU_MODULE_VERSION]) if test -z "$ICU_MODULE_VERSION" From 82c8d9a9cbdee3ca9195c4dbcde258c58f7e7346 Mon Sep 17 00:00:00 2001 From: micronn Date: Sun, 25 Dec 2022 03:06:30 +0100 Subject: [PATCH 095/121] Make it work for HP Ultrium 3000 on FreeBSD (#379) IBM drives seem to ignore PF bit in mode select, but HP drives want it set. --- src/tape_drivers/freebsd/cam/cam_tc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index e1c5bc65..26ccc735 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -1990,7 +1990,7 @@ int camtape_modeselect(void *device, unsigned char *buf, const size_t size) /*retries*/ 1, /*cbfcnp*/ NULL, /*tag_action*/ MSG_SIMPLE_Q_TAG, - /*scsi_page_fmt*/ 0, + /*scsi_page_fmt*/ 1, /*save_pages*/ 0, /*param_buf*/ buf, /*param_len*/ size, From f95a9a4463a0e3dbb0b69aa57324a78e4eb80322 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Mon, 13 Mar 2023 11:30:03 +0900 Subject: [PATCH 096/121] Add a few headers to be installed (#386) --- src/Makefile.am | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 4469ba8a..7afe5ef7 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -91,9 +91,7 @@ noinst_HEADERS = \ libltfs/pathname.h \ libltfs/periodic_sync.h \ libltfs/xattr.h \ - libltfs/xml.h \ libltfs/xml_libltfs.h \ - libltfs/arch/errormap.h \ libltfs/arch/filename_handling.h \ libltfs/arch/uuid_internal.h \ libltfs/arch/version.h @@ -119,9 +117,11 @@ nobase_pkginclude_HEADERS = \ libltfs/uthash_ext.h \ libltfs/plugin.h \ libltfs/tape.h \ + libltfs/xml.h \ libltfs/arch/signal_internal.h \ libltfs/arch/arch_info.h \ libltfs/arch/time_internal.h \ + libltfs/arch/errormap.h \ tape_drivers/ibm_tape.h \ tape_drivers/spc_op_codes.h \ tape_drivers/ssc_op_codes.h \ From b80b20c5995111d6a0caa670abb40543eb72d4f9 Mon Sep 17 00:00:00 2001 From: Missael Palacios Date: Fri, 19 May 2023 13:41:46 -0600 Subject: [PATCH 097/121] Added warning to show fuse main error codes --- messages/bin_ltfs/root.txt | 1 + src/main.c | 13 ++++++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/messages/bin_ltfs/root.txt b/messages/bin_ltfs/root.txt index 5ff7b519..e6aa9066 100644 --- a/messages/bin_ltfs/root.txt +++ b/messages/bin_ltfs/root.txt @@ -159,6 +159,7 @@ root:table { 14120W:string { "Cannot access to directory %s, disabled index capture mode (%d)." } 14121I:string { "Index will be captured at %s at update" } 14122I:string { "Index will not be captured." } + 14123W:string { "Fuse main returned the error code: (%d)." } // 14150 - 14199 are reserved for LE+ diff --git a/src/main.c b/src/main.c index 86909b0a..4435fe6d 100644 --- a/src/main.c +++ b/src/main.c @@ -382,7 +382,7 @@ int ltfs_parse_options(void *priv_data, const char *arg, int key, struct fuse_ar struct ltfs_fuse_data *priv = (struct ltfs_fuse_data *) priv_data; const char *fuse_options[] = { "-f", "-d", "-s", NULL }; bool valid_fuse_option = false; - int i; + int i, ret; switch(key) { case KEY_VERSION: @@ -413,8 +413,12 @@ int ltfs_parse_options(void *priv_data, const char *arg, int key, struct fuse_ar default: if (! priv->first_parsing_pass) { fuse_opt_add_arg(outargs, "-h"); - if (priv->advanced_help) - fuse_main(outargs->argc, outargs->argv, <fs_ops, NULL); + if (priv->advanced_help){ + ret = fuse_main(outargs->argc, outargs->argv, <fs_ops, NULL); + if (ret != 0) { + ltfsmsg(LTFS_WARN, 14123W, ret) + } + } usage(outargs->argv[0], priv); exit(key == KEY_HELP ? 0 : 1); } @@ -1335,6 +1339,9 @@ int single_drive_main(struct fuse_args *args, struct ltfs_fuse_data *priv) ltfsmsg(LTFS_INFO, 14112I); ltfsmsg(LTFS_INFO, 14113I); ret = fuse_main(args->argc, args->argv, <fs_ops, priv); + if (ret != 0) { + ltfsmsg(LTFS_WARN, 14123W, ret); + } /* Setup signal handler again to terminate cleanly */ ret = ltfs_set_signal_handlers(); From cccf09e80794d5d2ad7bdddb904679240d82b627 Mon Sep 17 00:00:00 2001 From: Missael Palacios Date: Fri, 19 May 2023 13:49:10 -0600 Subject: [PATCH 098/121] Solving syntax error --- src/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main.c b/src/main.c index 4435fe6d..a4147c7e 100644 --- a/src/main.c +++ b/src/main.c @@ -416,7 +416,7 @@ int ltfs_parse_options(void *priv_data, const char *arg, int key, struct fuse_ar if (priv->advanced_help){ ret = fuse_main(outargs->argc, outargs->argv, <fs_ops, NULL); if (ret != 0) { - ltfsmsg(LTFS_WARN, 14123W, ret) + ltfsmsg(LTFS_WARN, 14123W, ret); } } usage(outargs->argv[0], priv); From 7edac12d46675550e9c1ea07d303d42ba2c35c0c Mon Sep 17 00:00:00 2001 From: Missael Palacios Date: Fri, 26 May 2023 13:47:49 -0600 Subject: [PATCH 099/121] Changed message of warning --- messages/bin_ltfs/root.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/messages/bin_ltfs/root.txt b/messages/bin_ltfs/root.txt index e6aa9066..043aea02 100644 --- a/messages/bin_ltfs/root.txt +++ b/messages/bin_ltfs/root.txt @@ -159,7 +159,7 @@ root:table { 14120W:string { "Cannot access to directory %s, disabled index capture mode (%d)." } 14121I:string { "Index will be captured at %s at update" } 14122I:string { "Index will not be captured." } - 14123W:string { "Fuse main returned the error code: (%d)." } + 14123W:string { "The main function of FUSE returned error (%d)." } // 14150 - 14199 are reserved for LE+ From a1f435fa4bfe3f986f90c2221895bf2ffdc57964 Mon Sep 17 00:00:00 2001 From: Missael Palacios Date: Mon, 19 Jun 2023 14:51:25 -0600 Subject: [PATCH 100/121] Reverting change on ltfs_parse_options --- src/main.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/src/main.c b/src/main.c index a4147c7e..48fa445e 100644 --- a/src/main.c +++ b/src/main.c @@ -382,7 +382,7 @@ int ltfs_parse_options(void *priv_data, const char *arg, int key, struct fuse_ar struct ltfs_fuse_data *priv = (struct ltfs_fuse_data *) priv_data; const char *fuse_options[] = { "-f", "-d", "-s", NULL }; bool valid_fuse_option = false; - int i, ret; + int i; switch(key) { case KEY_VERSION: @@ -413,12 +413,8 @@ int ltfs_parse_options(void *priv_data, const char *arg, int key, struct fuse_ar default: if (! priv->first_parsing_pass) { fuse_opt_add_arg(outargs, "-h"); - if (priv->advanced_help){ + if (priv->advanced_help) ret = fuse_main(outargs->argc, outargs->argv, <fs_ops, NULL); - if (ret != 0) { - ltfsmsg(LTFS_WARN, 14123W, ret); - } - } usage(outargs->argv[0], priv); exit(key == KEY_HELP ? 0 : 1); } From 9fcc77e42c8a523147af7fd57a17d436714dfe41 Mon Sep 17 00:00:00 2001 From: Missael Palacios Date: Mon, 19 Jun 2023 14:57:21 -0600 Subject: [PATCH 101/121] Reverting change --- src/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main.c b/src/main.c index 48fa445e..434d69ad 100644 --- a/src/main.c +++ b/src/main.c @@ -414,7 +414,7 @@ int ltfs_parse_options(void *priv_data, const char *arg, int key, struct fuse_ar if (! priv->first_parsing_pass) { fuse_opt_add_arg(outargs, "-h"); if (priv->advanced_help) - ret = fuse_main(outargs->argc, outargs->argv, <fs_ops, NULL); + fuse_main(outargs->argc, outargs->argv, <fs_ops, NULL); usage(outargs->argv[0], priv); exit(key == KEY_HELP ? 0 : 1); } From 178a782ffed151ff890b41cfdf56f7b8343d3c45 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 21 Jun 2023 20:12:03 +0900 Subject: [PATCH 102/121] Handle one-shot false `good` against TUR at failover process (#410) --- messages/tape_linux_sg/root.txt | 2 + src/tape_drivers/linux/sg/sg_tape.c | 83 ++++++++++++++++++++++++----- 2 files changed, 73 insertions(+), 12 deletions(-) diff --git a/messages/tape_linux_sg/root.txt b/messages/tape_linux_sg/root.txt index 9e2047aa..c674b83d 100644 --- a/messages/tape_linux_sg/root.txt +++ b/messages/tape_linux_sg/root.txt @@ -134,6 +134,8 @@ root:table { 30292I:string { "Changer %s was reserved from this node but failed to reserve from the current path." } 30293I:string { "Changer %s was reserved from another node (%s)." } 30294I:string { "Setting up timeout values from %s." } + 30295I:string { "Have unstable TUR response, start over (Cur = %d, Prev = %d)." } + 30296I:string { "Capturing a stable TUR at line %d." } 30392D:string { "Backend %s %s." } 30393D:string { "Backend %s: %d %s." } diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index 46517dcc..b77a5bf6 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -616,6 +616,39 @@ void _clear_por_raw(const int fd) } } +#define _get_stable_tur_response(p) _get_stable_tur_response_raw((p)->dev.fd) + +int _get_stable_tur_response_raw(const int fd) +{ + int i = 0, ret = -1, ret_tur = -1; + + do { + ret_tur = _raw_tur(fd); + if (i == 0) { + /* Keep first return code if it is not an unit attention */ + if (!IS_UNIT_ATTENTION(-ret_tur)) { + ret = ret_tur; + i++; + } + } else if (ret_tur == ret) { + /* Increment counter because it is same as previous response */ + i++; + } else { + /* TUR response is not stable, start over */ + ltfsmsg(LTFS_INFO, 30295I, ret_tur, ret); + if (IS_UNIT_ATTENTION(-ret_tur)) { + ret = -1; + i = 0; + } else { + ret = ret_tur; + i = 1; + } + } + } while (i < 3); + + return ret; +} + /* Forward reference */ int sg_get_device_list(struct tc_drive_info *buf, int count); int sg_reserve(void *device); @@ -770,7 +803,13 @@ static int _reconnect_device(void *device) /* Issue TUR and check reservation conflict happens or not */ _clear_por(priv); - ret = _raw_tur(priv->dev.fd); + + /* + * !!!!! This is a kind of work around to avoid to fetch false one-shot `good` here. + * Fetch result of TUR until 3 straight same result + */ + ltfsmsg(LTFS_INFO, 30296I, __LINE__); + ret = _get_stable_tur_response(priv); if (ret == -EDEV_RESERVATION_CONFLICT) { /* Select another path, recover reservation */ ltfsmsg(LTFS_INFO, 30269I, priv->drive_serial); @@ -785,23 +824,43 @@ static int _reconnect_device(void *device) } else { /* Read reservation information and print */ _clear_por(priv); - memset(&r_info, 0x00, sizeof(r_info)); - f_ret = _fetch_reservation_key(device, &r_info); - if (f_ret == -EDEV_NO_RESERVATION_HOLDER) { - /* Real POR may happens */ - ltfsmsg(LTFS_INFO, 30270I, priv->drive_serial); + + /* + * !!!!! This is the code just in case, check TUR response again and restore reservation + * if drive reports `reservation conflict`. + */ + ltfsmsg(LTFS_INFO, 30296I, __LINE__); + ret = _get_stable_tur_response(priv); + if (ret == -EDEV_RESERVATION_CONFLICT) { + /* Select another path, recover reservation */ + ltfsmsg(LTFS_INFO, 30269I, priv->drive_serial); _register_key(priv, priv->key); - ret = sg_reserve(device); + ret = _cdb_pro(device, PRO_ACT_PREEMPT_ABORT, PRO_TYPE_EXCLUSIVE, + priv->key, priv->key); if (!ret) { ltfsmsg(LTFS_INFO, 30272I, priv->drive_serial); _clear_por(priv); - ret = -EDEV_REAL_POWER_ON_RESET; + ret = -EDEV_NEED_FAILOVER; } } else { - /* Select same path */ - ltfsmsg(LTFS_INFO, 30271I, priv->drive_serial); - _clear_por(priv); - ret = -EDEV_NEED_FAILOVER; + memset(&r_info, 0x00, sizeof(r_info)); + f_ret = _fetch_reservation_key(device, &r_info); + if (f_ret == -EDEV_NO_RESERVATION_HOLDER) { + /* Real POR may happens */ + ltfsmsg(LTFS_INFO, 30270I, priv->drive_serial); + _register_key(priv, priv->key); + ret = sg_reserve(device); + if (!ret) { + ltfsmsg(LTFS_INFO, 30272I, priv->drive_serial); + _clear_por(priv); + ret = -EDEV_REAL_POWER_ON_RESET; + } + } else { + /* Select same path */ + ltfsmsg(LTFS_INFO, 30271I, priv->drive_serial); + _clear_por(priv); + ret = -EDEV_NEED_FAILOVER; + } } } From 6e4f79f13a4e5bd0f4c396fc18e9953a1caf08d1 Mon Sep 17 00:00:00 2001 From: micronn Date: Fri, 23 Jun 2023 08:07:35 +0200 Subject: [PATCH 103/121] Add support for FreeBSD to ltfs_ordered_copy (#380) --- src/utils/ltfs_ordered_copy | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/src/utils/ltfs_ordered_copy b/src/utils/ltfs_ordered_copy index d9809909..e777ff62 100755 --- a/src/utils/ltfs_ordered_copy +++ b/src/utils/ltfs_ordered_copy @@ -34,6 +34,7 @@ # OO_Copyright_END import sys +import errno import platform import os.path import argparse @@ -44,6 +45,11 @@ import threading from logging import getLogger, basicConfig, NOTSET, CRITICAL, ERROR, WARNING, INFO, DEBUG from collections import deque +def is_errno(num, names): + if not num in errno.errorcode: + return False + return errno.errorcode[num] in names + class CopyItem: """""" def __init__(self, src, dst, vea_pre, cp_attr, cp_xattr, logger): #initialization @@ -60,14 +66,14 @@ class CopyItem: def eval(self): #access to the extended attributes present in some operating systems/filesystems by xattr try: - self.vuuid = xattr.get(self.src, self.vea_pre + 'ltfs.volumeUUID') + self.vuuid = xattr.getxattr(self.src, self.vea_pre + 'ltfs.volumeUUID') except Exception as e: self.vuuid = '' return (self.vuuid, self.part, self.start) try: - self.part = xattr.get(self.src, self.vea_pre + 'ltfs.partition') - start_str = xattr.get(self.src, self.vea_pre + 'ltfs.startblock') + self.part = xattr.getxattr(self.src, self.vea_pre + 'ltfs.partition') + start_str = xattr.getxattr(self.src, self.vea_pre + 'ltfs.startblock') self.start = int(start_str) self.size = os.path.getsize(self.src) except Exception as e: @@ -89,12 +95,12 @@ class CopyItem: if self.cp_xattr: # Capture EAs of the source file src_attributes = {} - for key in xattr.list(self.src): - src_attributes[key] = xattr.get(self.src, key) + for key in xattr.listxattr(self.src): + src_attributes[key] = xattr.getxattr(self.src, key) # Set EAs of the destination file (_, filename) = os.path.split(self.src) for key in src_attributes: - xattr.set(self.dst, key, src_attributes[key]) + xattr.setxattr(self.dst, key, src_attributes[key]) else: #Only copy data shutil.copy(self.src, self.dst) except Exception as e: @@ -178,7 +184,7 @@ class CopyQueue: self.logger.log(NOTSET + 1, 'Making a directory {}'.format(d)) os.mkdir(d) except OSError as e: - if e.errno != 17: # Not EEXIST + if e.errno != errno.EEXIST: self.logger.error(str(e) + "\n") exit(1) @@ -251,7 +257,7 @@ LTFS_SIG_VEA='ltfs.softwareProduct' plat = platform.system() if plat == 'Linux': VEA_PREFIX='user.' -elif plat == 'Darwin': +elif plat == 'Darwin' or plat == 'FreeBSD': VEA_PREFIX='' else: sys.stderr.write("unsupported platform '{0}'\n".format(plat)) @@ -362,7 +368,7 @@ if args.recursive == False and len(args.SOURCE) == 1: logger.log(NOTSET + 1, 'Checking destination is LTFS or not') direct_write_threads = 8 try: - sig = xattr.get(args.DEST, VEA_PREFIX + LTFS_SIG_VEA) + sig = xattr.getxattr(args.DEST, VEA_PREFIX + LTFS_SIG_VEA) if sig.startswith(b"LTFS"): logger.info("Destination {0} is LTFS".format(args.DEST)) @@ -370,10 +376,10 @@ try: else: logger.info("Destination {0} is not LTFS\n".format(args.DEST)) except IOError as e: - if e.errno != 61 and e.errno != 93 and e.errno != 95: # Not ENODATA, ENOATTR, EOPNOTSUPP + if not is_errno(e.errno, ['ENODATA', 'ENOATTR', 'ENOTSUP', 'EOPNOTSUPP']): logger.error('Check destination (I/O):' + str(e)) exit(2) - if e.errno == 95 and args.p: + if is_errno(e.errno, ['ENOTSUP', 'EOPNOTSUPP']) and args.p: logger.warning("{0} does not support xattr. Cannot use -p. Attributes will not be preserved during copy.".format(args.DEST)) args.p = False logger.warning("Destination {0} is not LTFS".format(args.DEST)) From 2897a5a11ef093a44903301c1cd39625b680cadb Mon Sep 17 00:00:00 2001 From: Richard Goedeken Date: Tue, 27 Jun 2023 08:02:51 -0700 Subject: [PATCH 104/121] bugfix: cannot mount LTFS tapes under Rocky Linux with lin_tape driver (#399) * bugfix: all of the functions calling lin_tape_ibmtape_logsense() need to evaluate the return code according to the new semantic, which is negative for error, or positive page size for success * return rc instead of -1 from lin_tape_ibmtape_logsense() in case _sioc_stioc_command() returns with non-zero code --- .../linux/lin_tape/lin_tape_ibmtape.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c index f8c1448c..b232626f 100644 --- a/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c +++ b/src/tape_drivers/linux/lin_tape/lin_tape_ibmtape.c @@ -2205,7 +2205,7 @@ int lin_tape_ibmtape_format(void *device, TC_FORMAT_TYPE format, const char *vol * @param page page code of log sense * @param buf pointer to buffer to store log data * @param size length of the buffer - * @return 0 on success or a negative value on error + * @return Page length on success or a negative value on error. */ #define MAX_UINT16 (0x0000FFFF) @@ -2231,6 +2231,7 @@ int lin_tape_ibmtape_logsense(void *device, const uint8_t page, const uint8_t su if (rc != DEVICE_GOOD) { lin_tape_ibmtape_process_errors(device, rc, msg, "logsense page", true); + return rc; } else { memcpy(buf, log_page.data, size); } @@ -2263,7 +2264,7 @@ int lin_tape_ibmtape_remaining_capacity(void *device, struct tc_remaining_cap *c if (IS_LTO(priv->drive_type) && (DRIVE_GEN(priv->drive_type) == 0x05)) { /* Issue LogPage 0x31 */ rc = lin_tape_ibmtape_logsense(device, LOG_TAPECAPACITY, (uint8_t)0, logdata, LOGSENSEPAGE); - if (rc) { + if (rc < 0) { ltfsmsg(LTFS_INFO, 30457I, LOG_TAPECAPACITY, rc); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_REMAINCAP)); return rc; @@ -2303,7 +2304,7 @@ int lin_tape_ibmtape_remaining_capacity(void *device, struct tc_remaining_cap *c else { /* Issue LogPage 0x17 */ rc = lin_tape_ibmtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); - if (rc) { + if (rc < 0) { ltfsmsg(LTFS_INFO, 30457I, LOG_VOLUMESTATS, rc); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_REMAINCAP)); return rc; @@ -2926,7 +2927,7 @@ int lin_tape_ibmtape_get_cartridge_health(void *device, struct tc_cartridge_heal /* Issue LogPage 0x37 */ cart_health->tape_efficiency = UNSUPPORTED_CARTRIDGE_HEALTH; rc = lin_tape_ibmtape_logsense(device, LOG_PERFORMANCE, (uint8_t)0, logdata, LOGSENSEPAGE); - if (rc) + if (rc < 0) ltfsmsg(LTFS_INFO, 30461I, LOG_PERFORMANCE, rc, "get cart health"); else { for(i = 0; i < (int)((sizeof(perfstats)/sizeof(perfstats[0]))); i++) { /* BEAM: loop doesn't iterate - Use loop for future enhancement. */ @@ -2978,7 +2979,7 @@ int lin_tape_ibmtape_get_cartridge_health(void *device, struct tc_cartridge_heal cart_health->passes_middle = UNSUPPORTED_CARTRIDGE_HEALTH; rc = lin_tape_ibmtape_logsense(device, LOG_VOLUMESTATS, (uint8_t)0, logdata, LOGSENSEPAGE); - if (rc) + if (rc < 0) ltfsmsg(LTFS_INFO, 30461I, LOG_VOLUMESTATS, rc, "get cart health"); else { for(i = 0; i < (int)((sizeof(volstats)/sizeof(volstats[0]))); i++) { @@ -3075,7 +3076,7 @@ int lin_tape_ibmtape_get_tape_alert(void *device, uint64_t *tape_alert) /* Issue LogPage 0x2E */ ta = 0; rc = lin_tape_ibmtape_logsense(device, LOG_TAPE_ALERT, (uint8_t)0, logdata, LOGSENSEPAGE); - if (rc) + if (rc < 0) ltfsmsg(LTFS_INFO, 30461I, LOG_TAPE_ALERT, rc, "get tape alert"); else { for(i = 1; i <= 64; i++) { @@ -3416,7 +3417,7 @@ int lin_tape_ibmtape_get_eod_status(void *device, int part) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_ENTER(REQ_TC_GETEODSTAT)); /* Issue LogPage 0x17 */ rc = lin_tape_ibmtape_logsense(device, LOG_VOL_STATISTICS, (uint8_t)0, logdata, LOGSENSEPAGE); - if (rc) { + if (rc < 0) { ltfsmsg(LTFS_WARN, 30464W, LOG_VOL_STATISTICS, rc); ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_GETEODSTAT)); return EOD_UNKNOWN; @@ -3486,7 +3487,7 @@ int lin_tape_ibmtape_get_xattr(void *device, const char *name, char **buf) ((priv->fetch_sec_acq_loss_w + 60 < now.tv_sec) && priv->dirty_acq_loss_w)) { rc = lin_tape_ibmtape_logsense(device, LOG_PERFORMANCE, LOG_PERFORMANCE_CAPACITY_SUB, logdata, LOGSENSEPAGE); - if (rc) + if (rc < 0) ltfsmsg(LTFS_INFO, 30461I, LOG_PERFORMANCE, rc, "get xattr"); else { if (parse_logPage(logdata, PERF_ACTIVE_CQ_LOSS_W, ¶m_size, logbuf, 16)) { From 66f0b2839a3ada6f65f136015f50497dc6438fdb Mon Sep 17 00:00:00 2001 From: Richard Goedeken Date: Thu, 29 Jun 2023 08:40:53 -0700 Subject: [PATCH 105/121] Treat extended attribute ltfs.mediaPool.name like a virtual extended attribute. Allow both reading and writing, and store in MAM data. (#388) --- src/libltfs/xattr.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index 40544105..1588a619 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -266,7 +266,6 @@ static inline bool _xattr_is_worm_ea(const char *name) static inline bool _xattr_is_stored_vea(const char *name) { if (strcmp(name, "ltfs.spannedFileOffset") && - strcmp(name, "ltfs.mediaPool.name") && strcasestr(name, "ltfs.permissions.") != name && strcasestr(name, "ltfs.hash.") != name) { @@ -480,6 +479,7 @@ static bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_vo || ! strcmp(name, "ltfs.mediaIndexPartitionAvailableSpace") || ! strcmp(name, "ltfs.mediaEncrypted") || ! strcmp(name, "ltfs.mediaPool.additionalInfo") + || ! strcmp(name, "ltfs.mediaPool.name") || ! strcmp(name, "ltfs.driveEncryptionState") || ! strcmp(name, "ltfs.driveEncryptionMethod") /* Vendor specific EAs */ @@ -787,9 +787,18 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons ret = _xattr_get_cartridge_capacity(&cap, &cap.remaining_ip, &val, name, vol); } else if (! strcmp(name, "ltfs.mediaEncrypted")) { ret = xattr_get_string(tape_get_media_encrypted(vol->device), &val, name); + } else if (! strcmp(name, "ltfs.mediaPool.name")) { + char *tmp=NULL; + ret = tape_get_media_pool_info(vol, &val, &tmp); + if (tmp) + free(tmp); + if (ret < 0 || !val) + ret = -LTFS_NO_XATTR; } else if (! strcmp(name, "ltfs.mediaPool.additionalInfo")) { char *tmp=NULL; ret = tape_get_media_pool_info(vol, &tmp, &val); + if (tmp) + free(tmp); if (ret < 0 || !val) ret = -LTFS_NO_XATTR; } else if (! strcmp(name, "ltfs.driveEncryptionState")) { @@ -1225,6 +1234,8 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va free(v); } else if (! strcmp(name, "ltfs.mediaPool.additionalInfo")) { ret = tape_set_media_pool_info(vol, value, size, false); + } else if (! strcmp(name, "ltfs.mediaPool.name")) { + ret = tape_set_media_pool_info(vol, value, size, true); } else ret = -LTFS_NO_XATTR; @@ -1443,15 +1454,6 @@ int xattr_set(struct dentry *d, const char *name, const char *value, size_t size } } - if (!strcmp(name, "ltfs.mediaPool.name")) { - ret = tape_set_media_pool_info(vol, value, size, true); - if (ret < 0) { - releasewrite_mrsw(&d->meta_lock); - goto out_unlock; - } - write_idx = true; - } - /* Set extended attribute */ ret = xattr_do_set(d, name, value, size, xattr); if (ret < 0) { From a7916bf0b8d7264117341a5bcd06fb32a5c160f8 Mon Sep 17 00:00:00 2001 From: ochomenosocho <56462133+ochomenosocho@users.noreply.github.com> Date: Mon, 17 Jul 2023 11:12:28 -0600 Subject: [PATCH 106/121] Recursive protection to _take_dump() (#412) * added recursive check for DRIVE_DUMP error * Test commit for multithreading with pthread * Added join() to the pthread and change lock() for unlock() * Removed the destroy() functions for the mutex and setted the mutex to be global * Accepted new error messaages * Added recusivity error message * Fixed identation issues * Removed the unnecessary thead safety * added mutex for thread protection * test commit for mutex * test commit for mutex * test commit for mutex * test commit for mutex * test commit for mutex * test commit for mutex * test commit for mutex * test commit for mutex * test commit for mutex * mutex added with function to start it * mutex added with function to start it * Added recursive counter to sg_data structure and removed the mutex * A sum operator is changed to a binary operator OR _parse_logPage * Changed datatype from uint16_t to uint8_t from cast * Wrong variable call fixed. * Added null check to recursive_counter * Implemented some optimizations to _parse_logPage and changed _take_dump to work with priv->recursive_counter * Identation fixed. * Changed the conditions flow to the original version * changed datatype to recursive_counter --------- Co-authored-by: moisesFigueroa --- messages/tape_linux_sg/root.txt | 2 ++ src/tape_drivers/linux/sg/sg_tape.c | 24 +++++++++++++++++------- src/tape_drivers/linux/sg/sg_tape.h | 1 + 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/messages/tape_linux_sg/root.txt b/messages/tape_linux_sg/root.txt index c674b83d..5bfc632b 100644 --- a/messages/tape_linux_sg/root.txt +++ b/messages/tape_linux_sg/root.txt @@ -100,6 +100,7 @@ root:table { 30258W:string { "Cannot retrieve drive dump: failed to read buffer (%d)." } 30259W:string { "Cannot retrieve drive dump: failed to write to dump file (%d)." } 30260W:string { "Cannot retrieve drive dump: wrote %d bytes out, expected %d." } + 30261W:string { "Cannot retrieve drive dump: failed to write to communicate with drive. Tried (%d) times." } 30261I:string { "Taking drive dump in buffer." } 30262I:string { "Forcing drive dump." } 30263I:string { "%s returns %s (%d) %s." } @@ -136,6 +137,7 @@ root:table { 30294I:string { "Setting up timeout values from %s." } 30295I:string { "Have unstable TUR response, start over (Cur = %d, Prev = %d)." } 30296I:string { "Capturing a stable TUR at line %d." } + 30297W:string { "Cannot retrieve drive dump: failed to communicate with drive. Tried (%d) times." } 30392D:string { "Backend %s %s." } 30393D:string { "Backend %s: %d %s." } diff --git a/src/tape_drivers/linux/sg/sg_tape.c b/src/tape_drivers/linux/sg/sg_tape.c index b77a5bf6..86d93e95 100644 --- a/src/tape_drivers/linux/sg/sg_tape.c +++ b/src/tape_drivers/linux/sg/sg_tape.c @@ -103,6 +103,8 @@ struct sg_global_data global_data; #define TU_DEFAULT_TIMEOUT (60) #define MAX_RETRY (100) +#define MAX_TAKE_DUMP_ATTEMPTS (10) + /* Forward references (For keep function order to struct tape_ops) */ int sg_readpos(void *device, struct tc_position *pos); int sg_locate(void *device, struct tc_position dest, struct tc_position *pos); @@ -118,19 +120,17 @@ static inline int _parse_logPage(const unsigned char *logdata, const uint16_t param, uint32_t *param_size, unsigned char *buf, const size_t bufsize) { - uint16_t page_len, param_code, param_len; - uint32_t i; + const uint16_t page_len = ((uint16_t)logdata[2] << 8) | (uint16_t)logdata[3]; + uint16_t param_code, param_len; + uint32_t i = LOG_PAGE_HEADER_SIZE; uint32_t ret = -EDEV_INTERNAL_ERROR; - page_len = ((uint16_t)logdata[2] << 8) + (uint16_t)logdata[3]; - i = LOG_PAGE_HEADER_SIZE; - while(i < page_len) { - param_code = ((uint16_t)logdata[i] << 8) + (uint16_t)logdata[i+1]; + param_code = ((uint16_t)logdata[i] << 8) | (uint16_t)logdata[i+1]; param_len = (uint16_t)logdata[i + LOG_PAGE_PARAMSIZE_OFFSET]; - if(param_code == param) + if (param_code == param) { *param_size = param_len; if(bufsize < param_len){ @@ -143,6 +143,7 @@ static inline int _parse_logPage(const unsigned char *logdata, break; } } + i += param_len + LOG_PAGE_PARAM_OFFSET; } @@ -392,6 +393,13 @@ static int _take_dump(struct sg_data *priv, bool capture_unforced) time_t now; struct tm *tm_now; + /* To check if the function became recursive */ + if (priv->recursive_counter > MAX_TAKE_DUMP_ATTEMPTS) { + ltfsmsg(LTFS_WARN, 30297W, priv->recursive_counter); + return 0; + } + priv->recursive_counter++; + if (priv->vendor != VENDOR_IBM) return 0; @@ -426,6 +434,8 @@ static int _take_dump(struct sg_data *priv, bool capture_unforced) ltfs_profiler_add_entry(priv->profiler, NULL, TAPEBEND_REQ_EXIT(REQ_TC_TAKEDUMPDRV)); + priv->recursive_counter = 0; + return 0; } diff --git a/src/tape_drivers/linux/sg/sg_tape.h b/src/tape_drivers/linux/sg/sg_tape.h index 4d9de00e..2220d0f1 100644 --- a/src/tape_drivers/linux/sg/sg_tape.h +++ b/src/tape_drivers/linux/sg/sg_tape.h @@ -82,6 +82,7 @@ struct sg_data { struct timeout_tape *timeouts; /**< Timeout table */ struct tc_drive_info info; /**< Drive information */ FILE* profiler; /**< The file pointer for profiler */ + int recursive_counter; /**< Recursive counter for take dump */ }; struct sg_global_data { From 7681d4d53f45def3a682c3752235a2ad7dfeb435 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 25 Aug 2023 00:15:18 +0900 Subject: [PATCH 107/121] Fix a memory leak at `tape_read_attr()` (#415) --- src/libltfs/tape.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/libltfs/tape.c b/src/libltfs/tape.c index 5c25c1fd..4de147a9 100644 --- a/src/libltfs/tape.c +++ b/src/libltfs/tape.c @@ -3553,6 +3553,8 @@ int tape_read_attr(struct device_data *dev, const tape_partition_t part, ret = len; } + free(inner_buf); + return ret; } From b9cf7674cea45b9893a1ccd5e66018daca097e12 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Julio=20C=C3=A9sar=20Cel=C3=B3n=20Altamirano?= <37186598+juliocelon@users.noreply.github.com> Date: Thu, 31 Aug 2023 09:48:55 -0600 Subject: [PATCH 108/121] Rebranding from Spectrum Archive to Storage Archive and adding copyright year (#422) --- src/libltfs/LTFS-MIB.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/libltfs/LTFS-MIB.txt b/src/libltfs/LTFS-MIB.txt index 5f9f8227..2770eaec 100644 --- a/src/libltfs/LTFS-MIB.txt +++ b/src/libltfs/LTFS-MIB.txt @@ -8,9 +8,9 @@ ibmLTFS MODULE-IDENTITY LAST-UPDATED "201503200000Z" ORGANIZATION "International Business Machines Corp." CONTACT-INFO "IBM Support" - DESCRIPTION "This file defines IBM Spectrum Archive LE/SDE MIB. + DESCRIPTION "This file defines IBM Storage Archive LE/SDE MIB. Licensed Materials - Property of IBM - (C) Copyright IBM Corp. 2014, 2015" + (C) Copyright IBM Corp. 2014, 2023" ::= { ibmProd 248 } ibm OBJECT IDENTIFIER ::= { enterprises 2 } From 18ae70c3caae96caad687c18f763b719c3c0c31e Mon Sep 17 00:00:00 2001 From: Justin Green Date: Thu, 24 Aug 2023 19:58:08 -0600 Subject: [PATCH 109/121] Linux Jaguar 7 Tape and TS1170 Drive Support (#423) --- src/libltfs/tape_ops.h | 4 +- src/tape_drivers/generic/file/filedebug_tc.c | 17 +++++++ src/tape_drivers/ibm_tape.c | 51 +++++++++++++++++++- src/tape_drivers/tape_drivers.h | 7 ++- 4 files changed, 76 insertions(+), 3 deletions(-) diff --git a/src/libltfs/tape_ops.h b/src/libltfs/tape_ops.h index a2e3e967..deae04a2 100644 --- a/src/libltfs/tape_ops.h +++ b/src/libltfs/tape_ops.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2023 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -166,6 +166,7 @@ enum { TC_DC_JAG5 = 0x55, TC_DC_JAG5A = 0x56, TC_DC_JAG6 = 0x57, + TC_DC_JAG7 = 0x59, TC_DC_JAG1E = 0x71, TC_DC_JAG2E = 0x72, TC_DC_JAG3E = 0x73, @@ -173,6 +174,7 @@ enum { TC_DC_JAG5E = 0x75, TC_DC_JAG5AE = 0x76, TC_DC_JAG6E = 0x77, + TC_DC_JAG7E = 0x79, }; #define ALL_MEDIA_DENSITY 0 diff --git a/src/tape_drivers/generic/file/filedebug_tc.c b/src/tape_drivers/generic/file/filedebug_tc.c index 74d8ca8c..f997be72 100644 --- a/src/tape_drivers/generic/file/filedebug_tc.c +++ b/src/tape_drivers/generic/file/filedebug_tc.c @@ -1418,6 +1418,23 @@ static inline int _sanitize_tape(struct filedebug_data *state) ret = -EDEV_MEDIUM_FORMAT_ERROR; break; } + } else if (gen == DRIVE_GEN_JAG7) { + switch (state->conf.cart_type) { + case TC_MP_JE: + case TC_MP_JM: + case TC_MP_JF: + state->is_worm = false; + break; + case TC_MP_JV: + state->is_worm = true; + break; + default: + ltfsmsg(LTFS_INFO, 30086I, "TS1170", state->conf.cart_type); + state->is_worm = false; + state->unsupported_tape = true; + ret = -EDEV_MEDIUM_FORMAT_ERROR; + break; + } } else { ltfsmsg(LTFS_INFO, 30086I, "Unexpected Drive", state->conf.cart_type); state->is_worm = false; diff --git a/src/tape_drivers/ibm_tape.c b/src/tape_drivers/ibm_tape.c index 70e5c3c8..42a8714b 100644 --- a/src/tape_drivers/ibm_tape.c +++ b/src/tape_drivers/ibm_tape.c @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2023 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -64,6 +64,16 @@ #include "libltfs/ltfs_endian.h" DRIVE_DENSITY_SUPPORT_MAP jaguar_drive_density[] = { + /* TS1170 */ + { DRIVE_GEN_JAG7, TC_MP_JF, TC_DC_JAG7, MEDIUM_PERFECT_MATCH }, + { DRIVE_GEN_JAG7, TC_MP_JF, TC_DC_UNKNOWN, MEDIUM_PROBABLY_WRITABLE }, + { DRIVE_GEN_JAG7, TC_MP_JE, TC_DC_JAG6, MEDIUM_WRITABLE }, + { DRIVE_GEN_JAG7, TC_MP_JE, TC_DC_UNKNOWN, MEDIUM_WRITABLE }, + { DRIVE_GEN_JAG7, TC_MP_JV, TC_DC_JAG6, MEDIUM_WRITABLE }, + { DRIVE_GEN_JAG7, TC_MP_JV, TC_DC_UNKNOWN, MEDIUM_WRITABLE }, + { DRIVE_GEN_JAG7, TC_MP_JM, TC_DC_JAG6, MEDIUM_WRITABLE }, + { DRIVE_GEN_JAG7, TC_MP_JM, TC_DC_UNKNOWN, MEDIUM_WRITABLE }, + /* TS1160 */ { DRIVE_GEN_JAG6, TC_MP_JE, TC_DC_JAG6, MEDIUM_PERFECT_MATCH }, { DRIVE_GEN_JAG6, TC_MP_JE, TC_DC_UNKNOWN, MEDIUM_PROBABLY_WRITABLE }, @@ -141,6 +151,10 @@ DRIVE_DENSITY_SUPPORT_MAP jaguar_drive_density[] = { }; DRIVE_DENSITY_SUPPORT_MAP jaguar_drive_density_strict[] = { + /* TS1170 */ + { DRIVE_GEN_JAG7, TC_MP_JF, TC_DC_JAG7, MEDIUM_PERFECT_MATCH }, + { DRIVE_GEN_JAG7, TC_MP_JF, TC_DC_UNKNOWN, MEDIUM_PROBABLY_WRITABLE }, + /* TS1160 */ { DRIVE_GEN_JAG6, TC_MP_JE, TC_DC_JAG6, MEDIUM_PERFECT_MATCH }, { DRIVE_GEN_JAG6, TC_MP_JE, TC_DC_UNKNOWN, MEDIUM_PROBABLY_WRITABLE }, @@ -256,13 +270,16 @@ const unsigned char supported_cart[] = { TC_MP_JE, TC_MP_JV, TC_MP_JM, + TC_MP_JF, }; const unsigned char supported_density[] = { + TC_DC_JAG7E, TC_DC_JAG6E, TC_DC_JAG5AE, TC_DC_JAG5E, TC_DC_JAG4E, + TC_DC_JAG7, TC_DC_JAG6, TC_DC_JAG5A, TC_DC_JAG5, @@ -315,6 +332,8 @@ struct supported_device *ibm_supported_drives[] = { TAPEDRIVE( IBM_VENDOR_ID, "0359260F", VENDOR_IBM, DRIVE_TS1160, "[0359260F]" ), /* IBM TS1160 */ TAPEDRIVE( IBM_VENDOR_ID, "0359260E", VENDOR_IBM, DRIVE_TS1160, "[0359260E]" ), /* IBM TS1160 */ TAPEDRIVE( IBM_VENDOR_ID, "0359260S", VENDOR_IBM, DRIVE_TS1160, "[0359260S]" ), /* IBM TS1160 */ + TAPEDRIVE( IBM_VENDOR_ID, "0359270F", VENDOR_IBM, DRIVE_TS1170, "[0359270F]" ), /* IBM TS1170 */ + TAPEDRIVE( IBM_VENDOR_ID, "0359270S", VENDOR_IBM, DRIVE_TS1170, "[0359270S]" ), /* IBM TS1170 */ /* End of supported_devices */ NULL }; @@ -794,6 +813,26 @@ static struct _timeout_tape timeout_1160[] = { {-1, -1} }; +static struct _timeout_tape timeout_1170[] = { + { XCOPY, 176820 }, + { ERASE, 175900 }, + { FORMAT_MEDIUM, 3120 }, + { LOAD_UNLOAD, 900 }, + { LOCATE10, 2280 }, + { LOCATE16, 2240 }, + { READ, 2340 }, + { READ_BUFFER, 480 }, + { REWIND, 600 }, + { SEND_DIAGNOSTIC, 2280 }, + { SPACE6, 2280 }, + { SPACE16, 2240 }, + { VERIFY, 176820 }, + { WRITE, 1440 }, + { WRITE_BUFFER, 540 }, + { WRITE_FILEMARKS6, 1380 }, + {-1, -1} +}; + static int _create_table_tape(struct timeout_tape **result, struct _timeout_tape* base, struct _timeout_tape* override) @@ -882,6 +921,9 @@ int ibm_tape_init_timeout(struct timeout_tape** table, int type) case DRIVE_TS1160: ret = _create_table_tape(table, timeout_11x0, timeout_1160); break; + case DRIVE_TS1170: + ret = _create_table_tape(table, timeout_11x0, timeout_1170); + break; default: ret = _create_table_tape(table, timeout_lto, timeout_lto7_hh); break; @@ -930,6 +972,9 @@ static inline unsigned char _assume_cartridge_type(char product, char btype) case 'M': ctype = TC_MP_JM; break; + case 'F': + ctype = TC_MP_JF; + break; default: break; } @@ -1033,6 +1078,9 @@ char* ibm_tape_assume_cart_name(unsigned char type) case TC_MP_JM: name = "JM"; break; + case TC_MP_JF: + name = "JF"; + break; default: name = "L5"; break; @@ -1295,6 +1343,7 @@ bool ibm_tape_is_supported_firmware(int drive_type, const unsigned char * const case DRIVE_LTO7_HH: case DRIVE_TS1150: case DRIVE_TS1160: + case DRIVE_TS1170: default: break; } diff --git a/src/tape_drivers/tape_drivers.h b/src/tape_drivers/tape_drivers.h index fa797a18..8a43a1e0 100644 --- a/src/tape_drivers/tape_drivers.h +++ b/src/tape_drivers/tape_drivers.h @@ -3,7 +3,7 @@ ** OO_Copyright_BEGIN ** ** -** Copyright 2010, 2020 IBM Corp. All rights reserved. +** Copyright 2010, 2023 IBM Corp. All rights reserved. ** ** Redistribution and use in source and binary forms, with or without ** modification, are permitted provided that the following conditions @@ -199,6 +199,7 @@ enum { DRIVE_TS1150 = 0x1105, /* TS1150 */ DRIVE_TS1155 = 0x5105, /* TS1155 */ DRIVE_TS1160 = 0x1106, /* TS1160 */ + DRIVE_TS1170 = 0x1107, /* TS1170 */ }; enum { @@ -212,6 +213,7 @@ enum { DRIVE_GEN_JAG5 = 0x1005, DRIVE_GEN_JAG5A = 0x5005, DRIVE_GEN_JAG6 = 0x1006, + DRIVE_GEN_JAG7 = 0x1007, }; /* LTO cartridge type in mode page header */ @@ -256,6 +258,8 @@ enum { TC_MP_JE = 0x95, /* IBM TS11x0 JE cartridge */ TC_MP_JV = 0xA5, /* IBM TS11x0 JV cartridge */ TC_MP_JM = 0xB4, /* IBM TS11x0 JM cartridge */ + /* 6th gen */ + TC_MP_JF = 0x96, /* IBM TS11x0 JF cartridge */ }; #define IS_REFORMATTABLE_TAPE(t) \ @@ -270,6 +274,7 @@ enum { t == TC_MP_JE || \ t == TC_MP_JV || \ t == TC_MP_JM || \ + t == TC_MP_JF || \ t == TC_MP_LTO7D_CART ) #endif // __tape_drivers_h From 82024a90970494ecaeca87dba16adfda03b90704 Mon Sep 17 00:00:00 2001 From: Diamond Rivero Date: Tue, 28 Nov 2023 00:47:39 +0800 Subject: [PATCH 110/121] Fix `ltfs -o device_list` exit code (#429) --- src/libltfs/ltfs.c | 2 +- src/main.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index c085c2a5..f901be8f 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -4371,7 +4371,7 @@ int ltfs_print_device_list(struct tape_ops *ops) } } - ret = 0; + //ret = 0; return ret; } diff --git a/src/main.c b/src/main.c index 434d69ad..0add6121 100644 --- a/src/main.c +++ b/src/main.c @@ -708,7 +708,7 @@ int main(int argc, char **argv) if (priv->device_list) { ret = show_device_list(priv); ltfs_finish(); - return (ret != 0) ? 0 : 1; + return ret ? 1 : 0; } /* Validate sync option */ From f36304e66f06f53281fadbcebc35615b0419b3f0 Mon Sep 17 00:00:00 2001 From: Justin Green Date: Mon, 27 Nov 2023 09:47:17 -0700 Subject: [PATCH 111/121] Jag7 fix (#435) --- src/tape_drivers/generic/file/filedebug_tc.c | 5 ----- src/tape_drivers/ibm_tape.c | 6 ------ 2 files changed, 11 deletions(-) diff --git a/src/tape_drivers/generic/file/filedebug_tc.c b/src/tape_drivers/generic/file/filedebug_tc.c index f997be72..b0ad4056 100644 --- a/src/tape_drivers/generic/file/filedebug_tc.c +++ b/src/tape_drivers/generic/file/filedebug_tc.c @@ -1420,14 +1420,9 @@ static inline int _sanitize_tape(struct filedebug_data *state) } } else if (gen == DRIVE_GEN_JAG7) { switch (state->conf.cart_type) { - case TC_MP_JE: - case TC_MP_JM: case TC_MP_JF: state->is_worm = false; break; - case TC_MP_JV: - state->is_worm = true; - break; default: ltfsmsg(LTFS_INFO, 30086I, "TS1170", state->conf.cart_type); state->is_worm = false; diff --git a/src/tape_drivers/ibm_tape.c b/src/tape_drivers/ibm_tape.c index 42a8714b..e618f91e 100644 --- a/src/tape_drivers/ibm_tape.c +++ b/src/tape_drivers/ibm_tape.c @@ -67,12 +67,6 @@ DRIVE_DENSITY_SUPPORT_MAP jaguar_drive_density[] = { /* TS1170 */ { DRIVE_GEN_JAG7, TC_MP_JF, TC_DC_JAG7, MEDIUM_PERFECT_MATCH }, { DRIVE_GEN_JAG7, TC_MP_JF, TC_DC_UNKNOWN, MEDIUM_PROBABLY_WRITABLE }, - { DRIVE_GEN_JAG7, TC_MP_JE, TC_DC_JAG6, MEDIUM_WRITABLE }, - { DRIVE_GEN_JAG7, TC_MP_JE, TC_DC_UNKNOWN, MEDIUM_WRITABLE }, - { DRIVE_GEN_JAG7, TC_MP_JV, TC_DC_JAG6, MEDIUM_WRITABLE }, - { DRIVE_GEN_JAG7, TC_MP_JV, TC_DC_UNKNOWN, MEDIUM_WRITABLE }, - { DRIVE_GEN_JAG7, TC_MP_JM, TC_DC_JAG6, MEDIUM_WRITABLE }, - { DRIVE_GEN_JAG7, TC_MP_JM, TC_DC_UNKNOWN, MEDIUM_WRITABLE }, /* TS1160 */ { DRIVE_GEN_JAG6, TC_MP_JE, TC_DC_JAG6, MEDIUM_PERFECT_MATCH }, From 5175c4cb409c79c9e8cb4d792ac28a7adc547e31 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 17 May 2024 15:37:23 +0900 Subject: [PATCH 112/121] Create hba_report.md --- .github/ISSUE_TEMPLATE/hba_report.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/hba_report.md diff --git a/.github/ISSUE_TEMPLATE/hba_report.md b/.github/ISSUE_TEMPLATE/hba_report.md new file mode 100644 index 00000000..ce497321 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/hba_report.md @@ -0,0 +1,15 @@ +--- +name: HBA report +description: Report a HBA that is working on the specific enviroment +assignees: piste-jp-ibm +lebels: HBA Report +title: HBA list request + +--- + +**Fill in the informatio of HBA and OS** + +| HBA type | Vendor | I/F type (FC, SAS etc) | OS | LTFS version or commit hash | Note | +| -------- | ------ | ---------------------- | --- | --------------------------- | ---- | +| | | | | | | + From 972c610fedf857da40fea93405ed015df7155f73 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 17 May 2024 15:40:38 +0900 Subject: [PATCH 113/121] Update hba_report.md --- .github/ISSUE_TEMPLATE/hba_report.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/hba_report.md b/.github/ISSUE_TEMPLATE/hba_report.md index ce497321..0884bfb3 100644 --- a/.github/ISSUE_TEMPLATE/hba_report.md +++ b/.github/ISSUE_TEMPLATE/hba_report.md @@ -1,6 +1,6 @@ --- name: HBA report -description: Report a HBA that is working on the specific enviroment +about: Report a HBA that is working on the specific enviroment assignees: piste-jp-ibm lebels: HBA Report title: HBA list request From cbed7efafe47f866d617876173921c8242f9d3b3 Mon Sep 17 00:00:00 2001 From: Kevin Bowling Date: Wed, 22 May 2024 02:33:26 -0700 Subject: [PATCH 114/121] freebsd: add missing semicolon in cam_tc (#426) --- src/tape_drivers/freebsd/cam/cam_tc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/tape_drivers/freebsd/cam/cam_tc.c b/src/tape_drivers/freebsd/cam/cam_tc.c index 26ccc735..d5ae85ec 100644 --- a/src/tape_drivers/freebsd/cam/cam_tc.c +++ b/src/tape_drivers/freebsd/cam/cam_tc.c @@ -1244,7 +1244,7 @@ static int camtape_get_next_block_to_xfer(void *device, struct tc_position *pos) camtape_process_errors(softc, rc, msg, "READPOS", true); else { pos->partition = ext_data.partition; - pos->block = scsi_8btou64(ext_data.last_object) + pos->block = scsi_8btou64(ext_data.last_object); ltfsmsg(LTFS_DEBUG, 30398D, "next-block-to-xfer", (unsigned long long) pos->block, 0, 0, softc->drive_serial); } From 102dcb41d1164387f71e27e4cd5ff6da13095c80 Mon Sep 17 00:00:00 2001 From: Krishnakant Kolkar <117534565+Krishhhnaa@users.noreply.github.com> Date: Wed, 22 May 2024 15:09:53 +0530 Subject: [PATCH 115/121] Fix the build break on FreeBSD 12 (#396) Co-authored-by: Atsushi Abe From 22edc441ad070528e412a302444db52de5fcc2d6 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Wed, 22 May 2024 18:40:15 +0900 Subject: [PATCH 116/121] Introduce the structures for incremental index for format spec 2.5 (#454) --- cd | 0 contrib/fssim/src/.gitignore | 1 + src/libltfs/Makefile.am | 1 + src/libltfs/fs.c | 93 ++++++++ src/libltfs/fs.h | 22 ++ src/libltfs/inc_journal.c | 450 +++++++++++++++++++++++++++++++++++ src/libltfs/inc_journal.h | 109 +++++++++ src/libltfs/ltfs.c | 62 +++++ src/libltfs/ltfs.h | 12 +- src/libltfs/ltfs_fsops.c | 139 ++++++----- src/libltfs/ltfs_fsops_raw.c | 4 +- src/libltfs/ltfs_internal.c | 22 +- src/libltfs/ltfs_internal.h | 3 +- src/libltfs/xattr.c | 18 +- 14 files changed, 863 insertions(+), 73 deletions(-) delete mode 100644 cd create mode 100644 contrib/fssim/src/.gitignore create mode 100644 src/libltfs/inc_journal.c create mode 100644 src/libltfs/inc_journal.h diff --git a/cd b/cd deleted file mode 100644 index e69de29b..00000000 diff --git a/contrib/fssim/src/.gitignore b/contrib/fssim/src/.gitignore new file mode 100644 index 00000000..6722cd96 --- /dev/null +++ b/contrib/fssim/src/.gitignore @@ -0,0 +1 @@ +*.xml diff --git a/src/libltfs/Makefile.am b/src/libltfs/Makefile.am index b3ae064d..9b9ab438 100644 --- a/src/libltfs/Makefile.am +++ b/src/libltfs/Makefile.am @@ -62,6 +62,7 @@ libltfs_la_SOURCES = \ config_file.c \ plugin.c \ periodic_sync.c \ + inc_journal.c \ arch/uuid_internal.c \ arch/filename_handling.c \ arch/time_internal.c \ diff --git a/src/libltfs/fs.c b/src/libltfs/fs.c index 01909508..38bd65a0 100644 --- a/src/libltfs/fs.c +++ b/src/libltfs/fs.c @@ -650,6 +650,99 @@ void fs_split_path(char *path, char **filename, size_t len) } } +int fs_path_clean(const char *path, struct ltfs_index *idx) +{ + int ret = 0; + struct dentry *d = NULL, *parent = NULL; + char *tmp_path, *start, *end; + + CHECK_ARG_NULL(path, -LTFS_NULL_ARG); + CHECK_ARG_NULL(idx, -LTFS_NULL_ARG); + + tmp_path = strdup(path); + if (! tmp_path) { + ltfsmsg(LTFS_ERR, 10001E, "fs_path_clean: tmp_path"); + return -LTFS_NO_MEMORY; + } + + /* Get a reference count on the root dentry. Either it will be returned immediately, or it + * will be disposed later after the first path lookup. */ + acquirewrite_mrsw(&idx->root->meta_lock); + ++idx->root->numhandles; + releasewrite_mrsw(&idx->root->meta_lock); + + if (idx->root->dirty) + idx->root->dirty = false; + + /* Did the caller ask for the root dentry? */ + if (*path == '\0' || ! strcmp(path, "/")) { + goto out; + } + + start = tmp_path + 1; + end = tmp_path; + d = idx->root; + + while (end) { + end = strstr(start, "/"); + if (end) + *end = '\0'; + + acquireread_mrsw(&d->contents_lock); + + if (parent) + releaseread_mrsw(&parent->contents_lock); + parent = d; + d = NULL; + + ret = fs_directory_lookup(parent, start, &d); + if (ret < 0 || ! d) { + releaseread_mrsw(&parent->contents_lock); + fs_release_dentry(parent); + + if (ret == 0) + ret = -LTFS_NO_DENTRY; + goto out; + } + + /* Release the parent if we aren't keeping any locks on it. + * Since we know 'parent' has a child (d), it's guaranteed that parent is still linked + * into the file system tree. Therefore, fs_release_dentry is just a fancy way of + * decrementing the handle count... so do that. */ + acquirewrite_mrsw(&parent->meta_lock); + --parent->numhandles; + releasewrite_mrsw(&parent->meta_lock); + + if (d->dirty) + d->dirty = false; + + if (end) + start = end + 1; + } + + releaseread_mrsw(&parent->contents_lock); + +out: + free(tmp_path); + + return ret; +} + +int fs_dir_clean(struct dentry *d) +{ + struct name_list *list_ptr = NULL, *list_tmp = NULL; + CHECK_ARG_NULL(d, -LTFS_NULL_ARG); + + if (d->isdir) { + HASH_ITER(hh, d->child_list, list_ptr, list_tmp) { + fs_dir_clean(list_ptr->d); + } + } else + d->dirty = false; + + return 0; +} + /** * Dispose a dentry and all resources used by it, including the struct dentry itself. * @param dentry dentry to dispose. diff --git a/src/libltfs/fs.h b/src/libltfs/fs.h index 781c79d9..4fe904b0 100644 --- a/src/libltfs/fs.h +++ b/src/libltfs/fs.h @@ -146,4 +146,26 @@ int fs_path_lookup(const char *path, int flags, struct dentry **dentry, struct l */ void fs_split_path(char *path, char **filename, size_t len); +/** + * Cleanup d->dirty flag in the provided path + * + * @param path Path to search for, in UTF-8 NFC. The path should be checked + * for invalid characters by the caller. This function validates the length of each + * path component. If path points to an empty string, this function returns the + * root dentry. + * @param idx LTFS index to search. + * @return 0 on success (dentry found), -LTFS_NO_DENTRY if no dentry was found, -LTFS_NAMETOOLONG + * if any component of the path is too long, or another negative value if an internal + * (unexpected) error occurs. + */ +int fs_path_clean(const char *path, struct ltfs_index *idx); + +/** + * Cleanup d->dirty flag under the provided directory (dentry) recursively + * + * @param d dentry structure to clean. Expect a directory. + * @return 0 on success, otherwise on error + */ +int fs_dir_clean(struct dentry *d); + #endif /* __fs_helper_h */ diff --git a/src/libltfs/inc_journal.c b/src/libltfs/inc_journal.c new file mode 100644 index 00000000..67712f92 --- /dev/null +++ b/src/libltfs/inc_journal.c @@ -0,0 +1,450 @@ +/* +** +** OO_Copyright_BEGIN +** +** +** Copyright 2010, 2024 IBM Corp. All rights reserved. +** +** Redistribution and use in source and binary forms, with or without +** modification, are permitted provided that the following conditions +** are met: +** 1. Redistributions of source code must retain the above copyright +** notice, this list of conditions and the following disclaimer. +** 2. Redistributions in binary form must reproduce the above copyright +** notice, this list of conditions and the following disclaimer in the +** documentation and/or other materials provided with the distribution. +** 3. Neither the name of the copyright holder nor the names of its +** contributors may be used to endorse or promote products derived from +** this software without specific prior written permission. +** +** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS'' +** AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +** IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +** ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE +** LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +** CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +** SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +** INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +** CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +** ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +** POSSIBILITY OF SUCH DAMAGE. +** +** +** OO_Copyright_END +** +************************************************************************************* +** +** COMPONENT NAME: IBM Linear Tape File System +** +** FILE NAME: inc_journal.c +** +** DESCRIPTION: Journal handling for incremental index +** +** AUTHORS: Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** +** ORIGINAL LOGIC: David Pease +** pease@coati.com +** +************************************************************************************* +*/ + +#include "ltfs.h" +#include "fs.h" +#include "inc_journal.h" + +static int _allocate_jentry(struct jentry **e, char *path, struct dentry* d) +{ + struct jentry *ent = NULL; + *e = NULL; + + ent = calloc(1, sizeof(struct jentry)); + if (!ent) { + ltfsmsg(LTFS_ERR, 11168E); + return -LTFS_NO_MEMORY; + } + + ent->id.full_path = path; + ent->id.uid = d->uid; + *e = ent; + + return 0; +} + +static int _dispose_jentry(struct jentry *ent) +{ + if (ent) { + if (ent->id.full_path) + free(ent->id.full_path); + free(ent); + } + + return 0; +} + +/** + * Handle created object in the tree. + * + * Caller need to grab vol->index->dirty_lock outside of this function. + * + * @param ppath parent path name of the object + * @param d dentry created + * @param vol pointer to the LTFS volume + */ +int incj_create(char *ppath, struct dentry *d, struct ltfs_volume *vol) +{ + int ret = -1, len = -1; + char *full_path = NULL; + struct jentry *ent = NULL; + struct jcreated_entry *jdir = NULL, *jd = NULL; + + /* Skip journal modification because of an error */ + if (vol->journal_err) { + return 0; + } + + /* Skip if an ancestor is already created in this session */ + TAILQ_FOREACH(jd, &vol->created_dirs, list) { + char* cp = jd->path; + if (strstr(ppath, cp) == ppath) { + return 0; + } + } + + /* Create full path of created object and jentry */ + len = asprintf(&full_path, "%s/%s", ppath, d->name.name); + if (len < 0) { + ltfsmsg(LTFS_ERR, 11168E); + vol->journal_err = true; + return -LTFS_NO_MEMORY; + } + + ret = _allocate_jentry(&ent, full_path, d); + if (ret < 0) { + vol->journal_err = true; + free(full_path); + return ret; + } + + ent->reason = CREATE; + ent->dentry = d; + + HASH_ADD(hh, vol->journal, id, sizeof(struct jentry), ent); + + if (d->isdir) { + jdir = calloc(1, sizeof(struct jcreated_entry)); + if (!jdir) { + ltfsmsg(LTFS_ERR, 11168E); + return -LTFS_NO_MEMORY; + } + + /* NOTE: Use same pointer of ent because it's life is same */ + jdir->path = ent->id.full_path; + + TAILQ_INSERT_TAIL(&vol->created_dirs, jdir, list); + } + + return 0; +} + +/** + * Handle modified file in the tree. + * + * Caller need to grab vol->index->dirty_lock outside of this function. + * + * @param path path name of the object + * @param d dentry to be modified (for recording uid) + * @param vol pointer to the LTFS volume + */ +int incj_modify(char *path, struct dentry *d, struct ltfs_volume *vol) +{ + int ret = -1; + struct jentry *ent = NULL; + struct jcreated_entry *jd = NULL; + + /* Skip journal modification because of an error */ + if (vol->journal_err) { + return 0; + } + + /* Skip journal modification because it is already existed */ + HASH_FIND(hh, vol->journal, &ent->id, sizeof(struct jentry), ent); + if (ent) { + return 0; + } + + /* Skip if an ancestor is already created in this session */ + TAILQ_FOREACH(jd, &vol->created_dirs, list) { + char *cp = jd->path; + if (strstr(path, cp) == path) { + return 0; + } + } + + ret = _allocate_jentry(&ent, path, d); + if (ret < 0) { + vol->journal_err = true; + return ret; + } + + ent->reason = MODIFY; + ent->dentry = d; + + HASH_ADD(hh, vol->journal, id, sizeof(struct jentry), ent); + + return 0; +} + +/** + * Handle deleted file in the tree. + * + * Caller need to grab vol->index->dirty_lock outside of this function. + * + * @param path path name of the object + * @param d dentry to be removed (for recording uid) + * @param vol pointer to the LTFS volume + */ +int incj_rmfile(char *path, struct dentry *d, struct ltfs_volume *vol) +{ + int ret = -1; + char *full_path = NULL; + struct journal_id id; + struct jentry *ent = NULL; + struct jcreated_entry *jd = NULL; + + /* Skip journal modification because of an error */ + if (vol->journal_err) { + return 0; + } + + id.full_path = path; + id.uid = d->uid; + HASH_FIND(hh, vol->journal, &id, sizeof(struct jentry), ent); + if (ent) { + if (ent->reason == CREATE) { + /* + * Remove the entry because this file is newly created and deleted + * in one incremental index session + */ + HASH_DEL(vol->journal, ent); + return 0; + } else if (ent->reason == MODIFY) { + /* + * Override the existing entry to DELETE_FILE record. + */ + ent->reason = DELETE_FILE; + ent->dentry = NULL; + return 0; + } + } + + /* Skip if an ancestor is already created in this session */ + TAILQ_FOREACH(jd, &vol->created_dirs, list) { + char *cp = jd->path; + if (strstr(path, cp) == path) { + return 0; + } + } + + /* Create full path of created object and jentry */ + full_path = strdup(path); + if (!full_path) { + ltfsmsg(LTFS_ERR, 11168E); + vol->journal_err = true; + return -LTFS_NO_MEMORY; + } + + ret = _allocate_jentry(&ent, full_path, d); + if (ret < 0) { + vol->journal_err = true; + return ret; + } + + ent->reason = DELETE_FILE; + + HASH_ADD(hh, vol->journal, id, sizeof(struct jentry), ent); + + return 0; +} + +/** + * Handle deleted directory in the tree. + * + * Caller need to grab vol->index->dirty_lock outside of this function. + * + * @param path path name of the object + * @param d dentry to be removed (for recording uid) + * @param vol pointer to the LTFS volume + */ +int incj_rmdir(char *path, struct dentry *d, struct ltfs_volume *vol) +{ + int ret = -1; + char *full_path = NULL; + struct jentry *ent = NULL, *je = NULL, *tmp = NULL; + struct jcreated_entry *jd = NULL, *dtmp = NULL; + + /* Skip journal modification because of an error */ + if (vol->journal_err) { + return 0; + } + + /* + * 1. Remove entry from created_dirs if created directory is removed in a same session + * 2. Skip if an ancestor is already created in this session + */ + TAILQ_FOREACH_SAFE(jd, &vol->created_dirs, list, dtmp) { + char *cp = jd->path; + if (strstr(path, cp) == path) { + if (!strcmp(path, cp)) { + TAILQ_REMOVE(&vol->created_dirs, jd, list); + /* + * NOTE: + * Do not free jd->path because it shall be freed into _dispose_jentry. + * jentry::id.full_path and jd->path points the same address + */ + } else { + return 0; + } + } + } + + /* Need to find existing children under this directory */ + HASH_ITER(hh, vol->journal, je, tmp) { + if (strstr(je->id.full_path, path) == je->id.full_path) { + HASH_DEL(vol->journal, je); + _dispose_jentry(je); + } + } + + /* Create full path of created object and jentry */ + full_path = strdup(path); + if (!full_path) { + ltfsmsg(LTFS_ERR, 11168E); + vol->journal_err = true; + return -LTFS_NO_MEMORY; + } + + ret = _allocate_jentry(&ent, full_path, d); + if (ret < 0) { + vol->journal_err = true; + return ret; + } + + ent->reason = DELETE_DIRECTORY; + + HASH_ADD(hh, vol->journal, id, sizeof(struct jentry), ent); + + return 0; +} + +/** + * Clear all entries into the incremental journal + */ +int incj_clear(struct ltfs_volume *vol) +{ + struct jentry *je = NULL, *tmp = NULL; + struct jcreated_entry *jd = NULL, *dtmp = NULL; + + TAILQ_FOREACH_SAFE(jd, &vol->created_dirs, list, dtmp) { + TAILQ_REMOVE(&vol->created_dirs, jd, list); + } + + HASH_ITER(hh, vol->journal, je, tmp) { + HASH_DEL(vol->journal, je); + _dispose_jentry(je); + } + + return 0; +} + +/** + * Sort function for dump incremental journal + */ +static int _by_path(const struct jentry *a, const struct jentry *b) +{ + int ret = 0; + + ret = strcmp(a->id.full_path, b->id.full_path); + if (!ret) { + if (a->id.uid > b->id.uid) + ret = 1; + else + ret = -1; + } + + return ret; +} + +static inline int dig_path(char *p, struct ltfs_index *idx) +{ + int ret = 0; + char *path; + + path = strdup(p); + if (! path) { + ltfsmsg(LTFS_ERR, 10001E, "dig_path: path"); + return -LTFS_NO_MEMORY; + } + + ret = fs_path_clean(path, idx); + + free(path); + + return ret; +} + +/** + * This is a function for debug. Print contents of the journal and the created + * directory list to stdout. + */ +void incj_dump(struct ltfs_volume *vol) +{ + char *prev_parent = NULL, *parent, *filename; + struct jcreated_entry *jd = NULL, *dtmp = NULL; + struct jentry *ent = NULL, *tmp = NULL; + char *reason[] = { "CREATE", "MODIFY", "DELFILE", "DELDIR" }; + + printf("===============================================================================\n"); + TAILQ_FOREACH_SAFE(jd, &vol->created_dirs, list, dtmp) { + printf("CREATED_DIR: %s\n", jd->path); + TAILQ_REMOVE(&vol->created_dirs, jd, list); + } + + printf("--------------------------------------------------------------------------------\n"); + HASH_SORT(vol->journal, _by_path); + HASH_ITER(hh, vol->journal, ent, tmp) { + printf("JOURNAL: %s, %llu, %s, ", ent->id.full_path, (unsigned long long)ent->id.uid, reason[ent->reason]); + if (!ent->dentry) + printf("no-dentry\n"); + else { + if (ent->dentry->isdir) { + printf("dir\n"); + if (ent->reason == CREATE) + fs_dir_clean(ent->dentry); + } else + printf("file\n"); + + parent= strdup(ent->id.full_path); + fs_split_path(parent, &filename, strlen(parent) + 1); + + if (prev_parent) { + if (strcmp(prev_parent, parent)) { + dig_path(parent, vol->index); + } + free(prev_parent); + } else { + dig_path(parent, vol->index); + } + prev_parent = parent; + ent->dentry->dirty = false; + } + + HASH_DEL(vol->journal, ent); + _dispose_jentry(ent); + } + + if (prev_parent) free(prev_parent); + + return; +} diff --git a/src/libltfs/inc_journal.h b/src/libltfs/inc_journal.h new file mode 100644 index 00000000..1b4b1b2f --- /dev/null +++ b/src/libltfs/inc_journal.h @@ -0,0 +1,109 @@ +/* +** +** OO_Copyright_BEGIN +** +** +** Copyright 2010, 2024 IBM Corp. All rights reserved. +** +** Redistribution and use in source and binary forms, with or without +** modification, are permitted provided that the following conditions +** are met: +** 1. Redistributions of source code must retain the above copyright +** notice, this list of conditions and the following disclaimer. +** 2. Redistributions in binary form must reproduce the above copyright +** notice, this list of conditions and the following disclaimer in the +** documentation and/or other materials provided with the distribution. +** 3. Neither the name of the copyright holder nor the names of its +** contributors may be used to endorse or promote products derived from +** this software without specific prior written permission. +** +** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS'' +** AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +** IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +** ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE +** LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +** CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +** SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +** INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +** CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +** ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +** POSSIBILITY OF SUCH DAMAGE. +** +** +** OO_Copyright_END +** +************************************************************************************* +** +** COMPONENT NAME: IBM Linear Tape File System +** +** FILE NAME: inc_journal.h +** +** DESCRIPTION: Journal handling for incremental index +** +** AUTHORS: Atsushi Abe +** IBM Tokyo Lab., Japan +** piste@jp.ibm.com +** +** ORIGINAL LOGIC: David Pease +** pease@coati.com +** +************************************************************************************* +*/ + +#include "queue.h" + +#ifndef __inc_journal_h__ +#define __inc_journal_h__ + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Enumeration of reasons for an entry + */ +enum journal_reason { + CREATE = 0, /**< Newly created, need to be 0 for debug */ + MODIFY, /**< Modified */ + DELETE_FILE, /**< File is deleted */ + DELETE_DIRECTORY, /**< Directory is deleted */ +}; + +/** + * Identifier of journal entry for handling multiple changes in one session + */ +struct journal_id { + char *full_path; /**< Full path name of the target */ + uint64_t uid; /**< i-node number of the target */ +}; + +/** + * Journal entry + */ +struct jentry { + struct journal_id id; /**< ID of the journal entry (key of the hash table) */ + enum journal_reason reason; /**< Reason of the entry */ + struct dentry *dentry; /**< Target dentry if required */ + UT_hash_handle hh; +}; + +/** + * Created directory list + */ +struct jcreated_entry { + TAILQ_ENTRY(jcreated_entry) list; /**< Pointers for linked list of requests */ + char *path; +}; + +int incj_create(char *ppath, struct dentry *d, struct ltfs_volume *vol); +int incj_modify(char *path, struct dentry *d, struct ltfs_volume *vol); +int incj_rmfile(char *path, struct dentry *d, struct ltfs_volume *vol); +int incj_rmdir(char *path, struct dentry *d, struct ltfs_volume *vol); +int incj_clear(struct ltfs_volume *vol); +void incj_dump(struct ltfs_volume *vol); + +#ifdef __cplusplus +} +#endif + +#endif /* __inc_journal_h__ */ diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index f901be8f..c6aae5d9 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -70,6 +70,7 @@ #include "xattr.h" #include "xml_libltfs.h" #include "label.h" +#include "inc_journal.h" #include "arch/version.h" #include "arch/filename_handling.h" #include "libltfs/arch/errormap.h" @@ -373,6 +374,9 @@ int ltfs_volume_alloc(const char *execname, struct ltfs_volume **volume) } } + newvol->journal = NULL; + TAILQ_INIT(&newvol->created_dirs); + *volume = newvol; return 0; @@ -2461,6 +2465,8 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) return ret; } + incj_clear(vol); /* Clear incremental journal data */ + bc_print = _get_barcode(vol); if (write_perm) { @@ -4572,3 +4578,59 @@ int ltfs_get_rao_list(char *path, struct ltfs_volume *vol) tape_device_unlock(vol->device); return ret; } + +static inline char* _lay_dirs(char *p, struct dentry *d) +{ + int ret = 0; + char *path = NULL; + + if (p) { + if (d->parent) + ret = asprintf(&path, "%s/%s", d->name.name, p); + else { + /* + * root dir doesn't have the parent and d->name.name is "/". + * So directory separator, '/', is not needed. + */ + ret = asprintf(&path, "%s%s", d->name.name, p); + } + free(p); + } else { + ret = asprintf(&path, "%s", d->name.name); + } + + if (ret < 0) + return NULL; + + if (d->parent) { + path = _lay_dirs(path, d->parent); + if (!path) { + /* Memory allocation error */ + return NULL; + } + } + + return path; +} + +/** + * Build full path from a dentry + */ +int ltfs_build_fullpath(char **dest, struct dentry *d) +{ + int ret = 0; + char *path = NULL; + + CHECK_ARG_NULL(dest, -LTFS_NULL_ARG); + + path = _lay_dirs(path, d); + if (path) { + *dest = path; + } else { + ltfsmsg(LTFS_ERR, 10001E, __FUNCTION__); + *dest = NULL; + ret = -LTFS_NO_MEMORY; + } + + return ret; +} diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index e34ffe3e..83b8a56e 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -339,7 +339,7 @@ struct dentry { /* Take the iosched_lock before accessing iosched_priv. */ void *iosched_priv; /**< I/O scheduler private data. */ - struct name_list *child_list; /* for hash search */ + struct name_list *child_list; /**< Child list for hash search */ }; struct tape_attr { @@ -458,10 +458,15 @@ struct ltfs_volume { struct tape_attr *t_attr; /**< Tape Attribute data */ mam_lockval lock_status; /**< Total volume lock status from t_attr->vollock and index->vollock */ struct ltfs_timespec first_locate; /**< Time to first locate */ - int file_open_count; /**< Number of opened files */ + int file_open_count; /**< Number of opened files */ - const char *work_directory; + /* Journal structure for incremental index */ + struct jentry *journal; /**< Journal for incremental index */ + TAILQ_HEAD(jcreated_struct, jcreated_entry) created_dirs; + bool journal_err; /**< Journal error flag, write a full index next time forcibly */ + /* Misc */ + const char *work_directory; /**< work directory for profiler data, dump etc.*/ }; struct ltfs_label { @@ -723,6 +728,7 @@ void ltfs_enable_livelink_mode(struct ltfs_volume *vol); int ltfs_profiler_set(uint64_t source, struct ltfs_volume *vol); int ltfs_get_rao_list(char *path, struct ltfs_volume *vol); +int ltfs_build_fullpath(char **dest, struct dentry *d); #ifdef __cplusplus } diff --git a/src/libltfs/ltfs_fsops.c b/src/libltfs/ltfs_fsops.c index efd37869..83d38a66 100644 --- a/src/libltfs/ltfs_fsops.c +++ b/src/libltfs/ltfs_fsops.c @@ -66,6 +66,7 @@ #include "dcache.h" #include "pathname.h" #include "index_criteria.h" +#include "inc_journal.h" #include "arch/time_internal.h" int ltfs_fsops_open(const char *path, bool open_write, bool use_iosched, struct dentry **d, @@ -367,9 +368,10 @@ int ltfs_fsops_create(const char *path, bool isdir, bool readonly, bool overwrit if (! isdir) ++vol->index->file_count; ltfs_set_index_dirty(false, false, vol->index); + incj_create(path_norm, d, vol); d->dirty = true; ltfs_mutex_unlock(&vol->index->dirty_lock); - vol->file_open_count ++; + vol->file_open_count++; *dentry = d; ret = 0; @@ -522,8 +524,12 @@ int ltfs_fsops_unlink(const char *path, ltfs_file_id *id, struct ltfs_volume *vo releasewrite_mrsw(&d->meta_lock); ltfs_mutex_lock(&vol->index->dirty_lock); - if (! d->isdir) + if (d->isdir) { + incj_rmdir(path_norm, d, vol); + } else { + incj_rmfile(path_norm, d, vol); --vol->index->file_count; + } ltfs_set_index_dirty(false, false, vol->index); ltfs_mutex_unlock(&vol->index->dirty_lock); @@ -548,7 +554,7 @@ int ltfs_fsops_rename(const char *from, const char *to, ltfs_file_id *id, struct int ret; char *from_norm = NULL, *to_norm = NULL; char *from_norm_copy = NULL, *to_norm_copy = NULL; - char *from_filename, *to_filename; + char *from_filename = NULL, *to_filename = NULL, *to_filename_incj = NULL; char *to_filename_copy = NULL, *to_filename_copy2 = NULL; struct dentry *fromdir = NULL, *todir = NULL; struct dentry *fromdentry = NULL, *todentry = NULL; @@ -587,14 +593,12 @@ int ltfs_fsops_rename(const char *from, const char *to, ltfs_file_id *id, struct goto out_free; } - if (dcache_initialized(vol)) { - from_norm_copy = strdup(from_norm); - to_norm_copy = strdup(to_norm); - if (! from_norm_copy || ! to_norm_copy) { - ltfsmsg(LTFS_ERR, 10001E, "ltfs_fsops_rename: file name copy"); - ret = -LTFS_NO_MEMORY; - goto out_free; - } + from_norm_copy = strdup(from_norm); + to_norm_copy = strdup(to_norm); + if (! from_norm_copy || ! to_norm_copy) { + ltfsmsg(LTFS_ERR, 10001E, "ltfs_fsops_rename: file name copy"); + ret = -LTFS_NO_MEMORY; + goto out_free; } /* Split paths into directory and file name */ @@ -868,6 +872,16 @@ int ltfs_fsops_rename(const char *from, const char *to, ltfs_file_id *id, struct fromdentry->dirty = true; + /* Process the incremental journal */ + if (fromdentry->isdir) + incj_rmdir(from_norm_copy, fromdentry, vol); + else + incj_rmfile(from_norm_copy, fromdentry, vol); + + fs_split_path(to_norm_copy, &to_filename_incj, strlen(to_norm_copy) + 1); + incj_create(to_norm_copy, fromdentry, vol); + + /* Release dentry of source */ if (! iosched_initialized(vol)) fs_release_dentry_unlocked(fromdentry); else @@ -1565,7 +1579,7 @@ int ltfs_fsops_utimens(struct dentry *d, const struct ltfs_timespec ts[2], struc d->platform_safe_name, (unsigned long long)d->uid, (unsigned long long)ts[0].tv_sec); get_current_timespec(&d->change_time); ltfs_set_index_dirty(true, true, vol->index); - d->dirty = true; + ltfs_set_dentry_dirty(d, vol); } if (d->modify_time.tv_sec != ts[1].tv_sec || d->modify_time.tv_nsec != ts[1].tv_nsec) { d->modify_time = ts[1]; @@ -1575,7 +1589,7 @@ int ltfs_fsops_utimens(struct dentry *d, const struct ltfs_timespec ts[2], struc d->platform_safe_name, (unsigned long long)d->uid, (unsigned long long)ts[1].tv_sec); get_current_timespec(&d->change_time); ltfs_set_index_dirty(true, false, vol->index); - d->dirty = true; + ltfs_set_dentry_dirty(d, vol); } if (dcache_initialized(vol)) dcache_flush(d, FLUSH_METADATA, vol); @@ -1642,7 +1656,7 @@ int ltfs_fsops_utimens_all(struct dentry *d, const struct ltfs_timespec ts[4], s d->platform_safe_name, (unsigned long long)d->uid, (unsigned long long)ts[3].tv_sec); isctime=true; ltfs_set_index_dirty(true, false, vol->index); - d->dirty = true; + ltfs_set_dentry_dirty(d, vol); } if (ts[0].tv_sec != 0 || ts[0].tv_nsec != 0) { d->access_time = ts[0]; @@ -1652,7 +1666,7 @@ int ltfs_fsops_utimens_all(struct dentry *d, const struct ltfs_timespec ts[4], s d->platform_safe_name, (unsigned long long)d->uid, (unsigned long long)ts[0].tv_sec); if(!isctime) get_current_timespec(&d->change_time); ltfs_set_index_dirty(true, true, vol->index); - d->dirty = true; + ltfs_set_dentry_dirty(d, vol); } if (ts[1].tv_sec != 0 || ts[1].tv_nsec != 0) { d->modify_time = ts[1]; @@ -1662,7 +1676,7 @@ int ltfs_fsops_utimens_all(struct dentry *d, const struct ltfs_timespec ts[4], s d->platform_safe_name, (unsigned long long)d->uid, (unsigned long long)ts[1].tv_sec); if(!isctime) get_current_timespec(&d->change_time); ltfs_set_index_dirty(true, false, vol->index); - d->dirty = true; + ltfs_set_dentry_dirty(d, vol); } if (ts[2].tv_sec != 0 || ts[2].tv_nsec != 0) { d->creation_time = ts[2]; @@ -1672,7 +1686,7 @@ int ltfs_fsops_utimens_all(struct dentry *d, const struct ltfs_timespec ts[4], s d->platform_safe_name, (unsigned long long)d->uid, (unsigned long long)ts[2].tv_sec); if(!isctime) get_current_timespec(&d->change_time); ltfs_set_index_dirty(true, false, vol->index); - d->dirty = true; + ltfs_set_dentry_dirty(d, vol); } if (dcache_initialized(vol)) @@ -1684,7 +1698,6 @@ int ltfs_fsops_utimens_all(struct dentry *d, const struct ltfs_timespec ts[4], s return 0; } - int ltfs_fsops_set_readonly(struct dentry *d, bool readonly, struct ltfs_volume *vol) { int ret; @@ -1710,6 +1723,7 @@ int ltfs_fsops_set_readonly(struct dentry *d, bool readonly, struct ltfs_volume d->readonly = readonly; get_current_timespec(&d->change_time); ltfs_set_index_dirty(true, false, vol->index); + ltfs_set_dentry_dirty(d, vol); if (dcache_initialized(vol)) dcache_flush(d, FLUSH_METADATA, vol); } @@ -2000,86 +2014,87 @@ int ltfs_fsops_readlink_path(const char* path, char* buf, size_t size, ltfs_file int ltfs_fsops_target_absolute_path(const char* link, const char* target, char* buf, size_t size ) { - char *work_buf, *target_buf, *temp_buf, *token, *next_token; // work buffer for string - int len=0,len2=0; // work variables for string length + char *work_buf, *target_buf, *temp_buf, *token, *next_token; /* work buffers for string */ + int len=0, len2=0; /* work variables for string length */ CHECK_ARG_NULL(link, -LTFS_NULL_ARG); CHECK_ARG_NULL(target, -LTFS_NULL_ARG); - // need to set message and return code + /* need to set message and return code */ if (link[0]!='/') { return -LTFS_BAD_ARG; } + /* Check input target string is already absolute path or not */ len2 = strlen(target); - // input target string is already absolute path if ( (target[0]=='/') && !strstr(target,"./" ) ) { if ( size < (size_t)len2+1) { return -LTFS_SMALL_BUFFER; } - strcpy( buf, target ); + strcpy(buf, target); return 0; } len=strlen(link); - work_buf = malloc(len+len2+1); + work_buf = malloc(len + len2 + 1); if (!work_buf) { return -LTFS_NO_MEMORY; } - target_buf = malloc(len2+1); + target_buf = malloc(len2 + 1); if (!target_buf) { - free( work_buf ); + free(work_buf); return -LTFS_NO_MEMORY; } - if ( target[0]=='/' ) { - temp_buf=strstr( target, "/." ); // get "/../ccc/target.txt" of "/aaa/../ccc/target.txt" - strcpy(target_buf,temp_buf+1); // copy "../ccc/target.txt" - len=strlen(target_buf)+1; - len=len2-len; - strncpy( work_buf, target, len ); // copy "/aaa" + if (target[0]=='/') { + temp_buf = strstr(target, "/."); /* get "/../ccc/target.txt" of "/aaa/../ccc/target.txt" */ + strcpy(target_buf, temp_buf + 1); /* copy "../ccc/target.txt" */ + len = strlen(target_buf) + 1; + len = len2 - len; + strncpy(work_buf, target, len); /* copy "/aaa" */ } else { - strcpy( work_buf, link ); - strcpy( target_buf, target ); - - // split link file name then get current directory - temp_buf=strrchr( work_buf, '/' ); // get "/link.txt" from "/aaa/bbb/link.txt" - len = len-strlen(temp_buf); // length of "/aaa/bbb" - } - - // split target path directory then modify current directory with target path information - token = strtok( target_buf,"/" ); // get ".." from "../ccc/target.txt" - while ( token ) { - next_token = strtok( NULL,"/" ); // if next_token is NULL then token is filename - if ( !next_token ) break; - if ( strcmp( token,".." )==0 ) { - work_buf[len]='\0'; // "/aaa/bbb\0link.txt" - temp_buf=strrchr( work_buf, '/' ); // get "/bbb" + strcpy(work_buf, link); + strcpy(target_buf, target); + + /* Split link file name then get current directory */ + temp_buf = strrchr(work_buf, '/'); /* get "/link.txt" from "/aaa/bbb/link.txt" */ + len -= strlen(temp_buf); /* length of "/aaa/bbb" */ + } + + /* Split target path directory then modify current directory with target path information */ + token = strtok(target_buf, "/"); /* get ".." from "../ccc/target.txt" */ + while (token) { + next_token = strtok(NULL, "/"); /* if next_token is NULL then token is filename */ + if (!next_token) + break; + if (strcmp(token, "..") == 0) { + work_buf[len] = '\0'; /* "/aaa/bbb\0link.txt" */ + temp_buf = strrchr(work_buf, '/' ); /* get "/bbb" */ if (!temp_buf) { - *buf = '\0'; // out of ltfs range + *buf = '\0'; /* out of ltfs range */ return 0; } - len = len-strlen(temp_buf); // length of "/aaa" - } else if ( strcmp( token, "." ) ) { // have directory name - work_buf[len]='/'; // put '/ 'as "/aaa/" - strncpy( work_buf+len+1, token, strlen(token)+1 ); // "/aaa/ccc\0" - len=strlen(work_buf); + len -= strlen(temp_buf); /* length of "/aaa" */ + } else if (strcmp(token, "." )) { /* have directory name */ + work_buf[len] = '/'; /* put '/ 'as "/aaa/" */ + strncpy(work_buf+len+1, token, strlen(token)+1); /* "/aaa/ccc\0" */ + len = strlen(work_buf); } token = next_token; } - work_buf[len]='/'; // put '/ 'as "/aaa/ccc/" - strncpy( work_buf+len+1, token, strlen(token)+1 ); // "/aaa/ccc/target.txt\0" + work_buf[len] = '/'; /* put '/ 'as "/aaa/ccc/" */ + strncpy(work_buf+len+1, token, strlen(token)+1); /* "/aaa/ccc/target.txt\0" */ - if ( size < strlen(work_buf)+1 ) { - free( work_buf ); - free( target_buf ); + if (size < strlen(work_buf) + 1) { + free(work_buf); + free(target_buf); return -LTFS_SMALL_BUFFER; } - strcpy( buf, work_buf ); - free( work_buf ); - free( target_buf ); + strcpy(buf, work_buf); + free(work_buf); + free(target_buf); return 0; } diff --git a/src/libltfs/ltfs_fsops_raw.c b/src/libltfs/ltfs_fsops_raw.c index 8a2b6ca8..d45a86de 100644 --- a/src/libltfs/ltfs_fsops_raw.c +++ b/src/libltfs/ltfs_fsops_raw.c @@ -398,7 +398,7 @@ int _ltfs_fsraw_add_extent_unlocked(struct dentry *d, struct extent_info *ext, b * No need to mark at this time but reserve this value for fueture release */ d->extents_dirty = true; - d->dirty = true; + ltfs_set_dentry_dirty(d, vol); releasewrite_mrsw(&d->meta_lock); ltfs_set_index_dirty(true, false, vol->index); @@ -793,12 +793,12 @@ int ltfs_fsraw_truncate(struct dentry *d, off_t length, struct ltfs_volume *vol) d->realsize = new_realsize; get_current_timespec(&d->modify_time); d->change_time = d->modify_time; + ltfs_set_dentry_dirty(d, vol); releasewrite_mrsw(&d->meta_lock); releasewrite_mrsw(&d->contents_lock); ltfs_set_index_dirty(true, false, vol->index); - d->dirty = true; releaseread_mrsw(&vol->lock); return 0; diff --git a/src/libltfs/ltfs_internal.c b/src/libltfs/ltfs_internal.c index 2f449f4e..8467f86d 100644 --- a/src/libltfs/ltfs_internal.c +++ b/src/libltfs/ltfs_internal.c @@ -65,6 +65,7 @@ #include "iosched.h" #include "ltfs_fsops.h" #include "xattr.h" +#include "inc_journal.h" /** * Allocate an empty LTFS index. @@ -1416,7 +1417,7 @@ int ltfs_write_index_conditional(char partition, struct ltfs_volume *vol) return ret; } -int ltfs_split_symlink( struct ltfs_volume *vol ) +int ltfs_split_symlink(struct ltfs_volume *vol) { size_t i, size; struct dentry *d, *workd; @@ -1527,3 +1528,22 @@ int ltfs_split_symlink( struct ltfs_volume *vol ) free(path); return ret; } + +int ltfs_set_dentry_dirty(struct dentry *d, struct ltfs_volume *vol) +{ + int ret = 0; + char *full_path = NULL; + + if (!d->dirty) { + ret = ltfs_build_fullpath(&full_path, d); + if (!ret) { + incj_modify(full_path, d, vol); + } else { + vol->journal_err = true; + } + } + + d->dirty = true; + + return ret; +} diff --git a/src/libltfs/ltfs_internal.h b/src/libltfs/ltfs_internal.h index e02365ea..714aea5e 100644 --- a/src/libltfs/ltfs_internal.h +++ b/src/libltfs/ltfs_internal.h @@ -85,6 +85,7 @@ int ltfs_seek_index(char partition, tape_block_t *eod_pos, tape_block_t *index_e bool *fm_after, bool *blocks_after, bool recover_symlink, struct ltfs_volume *vol); void _ltfs_last_ref(struct dentry *d, tape_block_t *dp_last, tape_block_t *ip_last, struct ltfs_volume *vol); -int ltfs_split_symlink( struct ltfs_volume *vol ); +int ltfs_split_symlink(struct ltfs_volume *vol); +int ltfs_set_dentry_dirty(struct dentry *d, struct ltfs_volume *vol); #endif /* __ltfs_internal_h__ */ diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index 1588a619..f1328910 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -66,6 +66,7 @@ #include "pathname.h" #include "tape.h" #include "ltfs_internal.h" +#include "inc_journal.h" #include "arch/time_internal.h" /* Helper functions for formatting virtual EA output */ @@ -209,7 +210,9 @@ static int _xattr_set_time(struct dentry *d, struct ltfs_timespec *out, const ch acquirewrite_mrsw(&d->meta_lock); *out = t; - d->dirty = true; + + ltfs_set_dentry_dirty(d, vol); + releasewrite_mrsw(&d->meta_lock); ltfs_set_index_dirty(true, false, vol->index); @@ -492,6 +495,7 @@ static bool _xattr_is_virtual(struct dentry *d, const char *name, struct ltfs_vo || ! strcmp(name, "ltfs.vendor.IBM.rao") || ! strcmp(name, "ltfs.vendor.IBM.logPage") || ! strcmp(name, "ltfs.vendor.IBM.mediaMAM") + || ! strcmp(name, "ltfs.vendor.IBM.dumpincj") || ! strncmp(name, "ltfs.vendor", strlen("ltfs.vendor"))) return true; } @@ -868,6 +872,10 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons ret = ltfs_mam(part, (unsigned char *)buf, buf_size, vol); + } else if (! strcmp(name, "ltfs.vendor.IBM.dumpincj")) { + incj_dump(vol); + ret = 0; + } else if (! strncmp(name, "ltfs.vendor", strlen("ltfs.vendor"))) { if (! strncmp(name + strlen("ltfs.vendor."), LTFS_VENDOR_NAME, strlen(LTFS_VENDOR_NAME))) { ret = _xattr_get_vendorunique_xattr(&val, name, vol); @@ -1472,9 +1480,11 @@ int xattr_set(struct dentry *d, const char *name, const char *value, size_t size } get_current_timespec(&d->change_time); - releasewrite_mrsw(&d->meta_lock); - d->dirty = true; + ltfs_set_index_dirty(true, false, vol->index); + ltfs_set_dentry_dirty(d, vol); + + releasewrite_mrsw(&d->meta_lock); if (write_idx) ret = ltfs_sync_index(SYNC_EA, false, vol); @@ -1734,8 +1744,8 @@ int xattr_remove(struct dentry *d, const char *name, struct ltfs_volume *vol) ltfsmsg(LTFS_INFO, 17238I, "appendonly", d->is_appendonly, d->name.name); } - d->dirty = true; ltfs_set_index_dirty(true, false, vol->index); + ltfs_set_dentry_dirty(d, vol); out_dunlk: _xattr_unlock_dentry(name, true, d, vol); From d19dfcf24d983960a18431b1f90c1147db0b242c Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Thu, 23 May 2024 20:45:46 +0900 Subject: [PATCH 117/121] Fix potential use after free issues (#462) --- src/libltfs/ltfstrace.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/libltfs/ltfstrace.c b/src/libltfs/ltfstrace.c index 7efa73b9..86567bfb 100644 --- a/src/libltfs/ltfstrace.c +++ b/src/libltfs/ltfstrace.c @@ -363,8 +363,8 @@ void ltfs_admin_function_trace_completed(uint32_t tid) static void ltfs_function_trace_destroy(void) { if (fs_tr_list) { - struct filesystem_trace_list *fsitem; - for (fsitem=fs_tr_list; fsitem != NULL; fsitem=fsitem->hh.next) { + struct filesystem_trace_list *fsitem, *tmp; + HASH_ITER(hh, fs_tr_list, fsitem, tmp) { destroy_mrsw(&fsitem->fn_entry->trace_lock); free(fsitem->fn_entry); free(fsitem); @@ -372,8 +372,8 @@ static void ltfs_function_trace_destroy(void) fs_tr_list = NULL; } if (admin_tr_list) { - struct admin_trace_list *aditem; - for (aditem=admin_tr_list; aditem != NULL; aditem=aditem->hh.next) { + struct admin_trace_list *aditem, *tmp; + HASH_ITER(hh, admin_tr_list, aditem, tmp) { destroy_mrsw(&aditem->fn_entry->trace_lock); free(aditem->fn_entry); free(aditem); @@ -381,8 +381,8 @@ static void ltfs_function_trace_destroy(void) admin_tr_list = NULL; } if (acomp) { - struct admin_completed_function_trace *tailq_item; - TAILQ_FOREACH (tailq_item, acomp, list) { + struct admin_completed_function_trace *tailq_item, *tmp; + TAILQ_FOREACH_SAFE(tailq_item, acomp, list, tmp) { destroy_mrsw(&tailq_item->trace_lock); free(tailq_item->fn_entry); free(tailq_item); From 06367df9ba1fff341f53b95b5b5a108b145f4d69 Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 24 May 2024 16:38:02 +0900 Subject: [PATCH 118/121] Change the `sync` behavior to fit the LTFS format Spec 2.5 (#461) --- src/iosched/unified.c | 1 + src/libltfs/ltfs.c | 69 ++++++++++++++++++++++++++++++---- src/libltfs/ltfs.h | 3 ++ src/libltfs/ltfs_fsops.c | 4 ++ src/libltfs/ltfs_internal.c | 10 +++-- src/libltfs/periodic_sync.c | 1 + src/libltfs/xattr.c | 74 ++++++++++++++++++++++--------------- src/ltfs_fuse.c | 5 ++- src/utils/ltfsck.c | 3 ++ 9 files changed, 129 insertions(+), 41 deletions(-) diff --git a/src/iosched/unified.c b/src/iosched/unified.c index 43cdacd2..af5265e7 100644 --- a/src/iosched/unified.c +++ b/src/iosched/unified.c @@ -2291,6 +2291,7 @@ int _unified_write_index_after_perm(int write_ret, struct unified_data *priv) return ret; } + ltfs_set_commit_message_reason(SYNC_WRITE_PERM, priv->vol); ret = ltfs_write_index(ltfs_ip_id(priv->vol), SYNC_WRITE_PERM, priv->vol); return ret; diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index c6aae5d9..40624c6e 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1779,6 +1779,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec } INTERRUPTED_GOTO(ret, out_unlock); ltfsmsg(LTFS_INFO, 11022I); + ltfs_set_commit_message_reason(SYNC_RECOVERY, vol); ret = ltfs_write_index(vol->label->partid_ip, SYNC_RECOVERY, vol); if (ret < 0) goto out_unlock; @@ -1864,13 +1865,6 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec if (vol->index->uid_number == 0) ltfsmsg(LTFS_WARN, 11307W, vol->label->vol_uuid); - /* Clear the commit message so it doesn't carry over from the previous session */ - /* TODO: is this the right place to clear the commit message? */ - if (vol->index->commit_message) { - free(vol->index->commit_message); - vol->index->commit_message = NULL; - } - /* If we reach this point, both partitions end in an index file. */ vol->ip_index_file_end = true; vol->dp_index_file_end = true; @@ -2535,6 +2529,7 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) write_perm = true; reason = SYNC_WRITE_PERM; + ltfs_set_commit_message_reason(SYNC_WRITE_PERM, vol); } /* ignore return value: we want to keep trying even if, e.g., the DP fills up */ } @@ -3137,6 +3132,7 @@ int ltfs_format_tape(struct ltfs_volume *vol, int density_code, bool destructive if (ret < 0) return ret; ltfsmsg(LTFS_INFO, 11278I, vol->label->partid_dp); /* "Writing Index to ..." */ + ltfs_set_commit_message_reason(SYNC_FORMAT, vol); ret = ltfs_write_index(vol->label->partid_dp, SYNC_FORMAT, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 11279E, vol->label->partid_dp, ret); @@ -3633,6 +3629,7 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) * For now, write the same index on IP and return an error against a sync * request. */ + ltfs_set_commit_message_reason(SYNC_WRITE_PERM, vol); ret_r = ltfs_write_index(ltfs_ip_id(vol), SYNC_WRITE_PERM, vol); if (!ret_r) { ltfsmsg(LTFS_INFO, 11344I, bc_print); @@ -4634,3 +4631,61 @@ int ltfs_build_fullpath(char **dest, struct dentry *d) return ret; } + +/** + * Update commit message of index to the prefixed message based on sync reason. + * @param reason sync reason defined in ltfs.h + * @param vol LTFS colume + */ +void ltfs_set_commit_message_reason(char *reason, struct ltfs_volume *vol) +{ + ltfs_mutex_lock(&vol->index->dirty_lock); + ltfs_set_commit_message_reason_unlocked(reason, vol); + ltfs_mutex_unlock(&vol->index->dirty_lock); + + return; +} + +/** + * Update commit message of index to the prefixed message based on sync reason. Caller need to + * take index->dirty_lock before calling this function. + * @param reason sync reason defined in ltfs.h + * @param vol LTFS volume + */ +void ltfs_set_commit_message_reason_unlocked(char *reason, struct ltfs_volume *vol) +{ + bool dirty = false; + int ret = 0; + char *str_now = NULL, *msg = NULL; + struct ltfs_timespec now; + + if (!vol->index->dirty) { + /* Do nothing because no update was made */ + goto out; + } + + if (vol->index->commit_message) { + free(vol->index->commit_message); + dirty = true; + } + + ret = get_current_timespec(&now); + if (ret) + goto out; + + ret = xml_format_time(now, &str_now); + if (ret) + goto out; + + ret = asprintf(&msg, "%s - %s", reason, str_now); + if (ret) { + vol->index->commit_message = msg; + dirty = true; + } + +out: + if (dirty) + ltfs_set_index_dirty(false, false, vol->index); + + return; +} diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index 83b8a56e..18f4b2dd 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -730,6 +730,9 @@ int ltfs_profiler_set(uint64_t source, struct ltfs_volume *vol); int ltfs_get_rao_list(char *path, struct ltfs_volume *vol); int ltfs_build_fullpath(char **dest, struct dentry *d); +void ltfs_set_commit_message_reason(char *reason, struct ltfs_volume *vol); +void ltfs_set_commit_message_reason_unlocked(char *reason, struct ltfs_volume *vol); + #ifdef __cplusplus } #endif diff --git a/src/libltfs/ltfs_fsops.c b/src/libltfs/ltfs_fsops.c index 83d38a66..be046720 100644 --- a/src/libltfs/ltfs_fsops.c +++ b/src/libltfs/ltfs_fsops.c @@ -2106,6 +2106,10 @@ int ltfs_fsops_volume_sync(char *reason, struct ltfs_volume *vol) if (ret < 0) return ret; + ltfs_mutex_lock(&vol->index->dirty_lock); + ltfs_set_commit_message_reason_unlocked(reason, vol); + ltfs_mutex_unlock(&vol->index->dirty_lock); + ret = ltfs_sync_index(reason, true, vol); return ret; diff --git a/src/libltfs/ltfs_internal.c b/src/libltfs/ltfs_internal.c index 8467f86d..cbd78c74 100644 --- a/src/libltfs/ltfs_internal.c +++ b/src/libltfs/ltfs_internal.c @@ -1323,6 +1323,7 @@ int ltfs_check_medium(bool fix, bool deep, bool recover_extra, bool recover_syml } } /* write to data partition if it doesn't end in an index file */ + ltfs_set_commit_message_reason(SYNC_RECOVERY, vol); if (! dp_have_index || dp_blocks_after) { ltfsmsg(LTFS_INFO, 17259I, "DP", vol->index->selfptr.partition, (unsigned long long)vol->index->selfptr.block); ret = ltfs_write_index(vol->label->partid_dp, SYNC_RECOVERY, vol); @@ -1407,12 +1408,15 @@ int ltfs_write_index_conditional(char partition, struct ltfs_volume *vol) CHECK_ARG_NULL(vol, -LTFS_NULL_ARG); - if (partition == ltfs_ip_id(vol) && ! vol->ip_index_file_end) + if (partition == ltfs_ip_id(vol) && ! vol->ip_index_file_end) { + ltfs_set_commit_message_reason(SYNC_CASCHE_PRESSURE, vol); ret = ltfs_write_index(partition, SYNC_CASCHE_PRESSURE, vol); - else if (partition == ltfs_dp_id(vol) && + } else if (partition == ltfs_dp_id(vol) && (! vol->dp_index_file_end || - (vol->ip_index_file_end && vol->index->selfptr.partition == ltfs_ip_id(vol)))) + (vol->ip_index_file_end && vol->index->selfptr.partition == ltfs_ip_id(vol)))) { + ltfs_set_commit_message_reason(SYNC_CASCHE_PRESSURE, vol); ret = ltfs_write_index(partition, SYNC_CASCHE_PRESSURE, vol); + } return ret; } diff --git a/src/libltfs/periodic_sync.c b/src/libltfs/periodic_sync.c index 33d71ae6..f11740c7 100644 --- a/src/libltfs/periodic_sync.c +++ b/src/libltfs/periodic_sync.c @@ -101,6 +101,7 @@ ltfs_thread_return periodic_sync_thread(void* data) ltfsmsg(LTFS_WARN, 17063W, __FUNCTION__); } + ltfs_set_commit_message_reason(SYNC_PERIODIC, priv->vol); ret = ltfs_sync_index(SYNC_PERIODIC, true, priv->vol); if (ret < 0) { ltfsmsg(LTFS_INFO, 11030I, ret); diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index f1328910..b223edd7 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -881,6 +881,7 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons ret = _xattr_get_vendorunique_xattr(&val, name, vol); } } else if (! strcmp(name, "ltfs.sync")) { + ltfs_set_commit_message_reason(SYNC_EA, vol); ret = ltfs_sync_index(SYNC_EA, false, vol); } } @@ -915,9 +916,8 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va { int ret = 0; - if (! strcmp(name, "ltfs.sync") && d == vol->index->root) - ret = ltfs_sync_index(SYNC_EA, false, vol); - else if (! strcmp(name, "ltfs.commitMessage") && d == vol->index->root) { + if ((! strcmp(name, "ltfs.commitMessage") || ! strcmp(name, "ltfs.sync")) + && d == vol->index->root) { char *value_null_terminated, *new_value; if (size > INDEX_MAX_COMMENT_LEN) { @@ -926,38 +926,49 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va } ltfs_mutex_lock(&vol->index->dirty_lock); - if (! value || ! size) { - /* Clear the current comment field */ - if (vol->index->commit_message) { - free(vol->index->commit_message); - vol->index->commit_message = NULL; - } + if (! vol->index->dirty) { + /* Do nothing because index is clean */ + ret = 0; } else { - value_null_terminated = malloc(size + 1); - if (! value_null_terminated) { - ltfsmsg(LTFS_ERR, 10001E, "_xattr_set_virtual: commit_message"); - ltfs_mutex_unlock(&vol->index->dirty_lock); - return -LTFS_NO_MEMORY; - } - memcpy(value_null_terminated, value, size); - value_null_terminated[size] = '\0'; + if (! value || ! size) { + /* Clear the current comment field */ + if (vol->index->commit_message) { + free(vol->index->commit_message); + vol->index->commit_message = NULL; + } + } else { + value_null_terminated = malloc(size + 1); + if (! value_null_terminated) { + ltfsmsg(LTFS_ERR, 10001E, "_xattr_set_virtual: commit_message"); + ltfs_mutex_unlock(&vol->index->dirty_lock); + return -LTFS_NO_MEMORY; + } + memcpy(value_null_terminated, value, size); + value_null_terminated[size] = '\0'; - ret = pathname_format(value_null_terminated, &new_value, false, true); - free(value_null_terminated); - if (ret < 0) { - ltfs_mutex_unlock(&vol->index->dirty_lock); - return ret; + ret = pathname_format(value_null_terminated, &new_value, false, true); + free(value_null_terminated); + if (ret < 0) { + /* Try to sync index even if the value is not valid */ + ltfs_set_commit_message_reason_unlocked(SYNC_EA, vol); + ltfs_mutex_unlock(&vol->index->dirty_lock); + + ret = ltfs_sync_index(SYNC_EA, false, vol); + return ret; + } + ret = 0; + + /* Update THE commit message in the index */ + if (vol->index->commit_message) + free(vol->index->commit_message); + vol->index->commit_message = new_value; } - ret = 0; - /* Update the commit message in the index */ - if (vol->index->commit_message) - free(vol->index->commit_message); - vol->index->commit_message = new_value; + ltfs_set_index_dirty(false, false, vol->index); } - ltfs_set_index_dirty(false, false, vol->index); ltfs_mutex_unlock(&vol->index->dirty_lock); + ret = ltfs_sync_index(SYNC_EA, false, vol); } else if (! strcmp(name, "ltfs.volumeName") && d == vol->index->root) { char *value_null_terminated, *new_value; @@ -1228,6 +1239,8 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va vol->lock_status = new; ltfs_set_index_dirty(false, false, vol->index); + ltfs_set_commit_message_reason_unlocked(SYNC_ADV_LOCK, vol); + ret = ltfs_sync_index(SYNC_ADV_LOCK, false, vol); ret = tape_device_lock(vol->device); if (ret < 0) { @@ -1486,9 +1499,10 @@ int xattr_set(struct dentry *d, const char *name, const char *value, size_t size releasewrite_mrsw(&d->meta_lock); - if (write_idx) + if (write_idx) { + ltfs_set_commit_message_reason(SYNC_EA, vol); ret = ltfs_sync_index(SYNC_EA, false, vol); - else + } else ret = 0; out_unlock: diff --git a/src/ltfs_fuse.c b/src/ltfs_fuse.c index e1347540..00aaf3bc 100644 --- a/src/ltfs_fuse.c +++ b/src/ltfs_fuse.c @@ -450,8 +450,10 @@ int ltfs_fuse_release(const char *path, struct fuse_file_info *fi) open_write = (((fi->flags & O_WRONLY) == O_WRONLY) || ((fi->flags & O_RDWR) == O_RDWR)); ret = ltfs_fsops_close(file->file_info->dentry_handle, dirty, open_write, true, priv->data); - if (write_index) + if (write_index) { + ltfs_set_commit_message_reason(SYNC_CLOSE, priv->data); ltfs_sync_index(SYNC_CLOSE, true, priv->data); + } _file_close(file->file_info, priv); _free_ltfs_file_handle(file); @@ -1167,6 +1169,7 @@ void ltfs_fuse_umount(void *userdata) if (kmi_initialized(priv->data)) kmi_destroy(priv->data); + ltfs_set_commit_message_reason(SYNC_UNMOUNT, priv->data); ltfs_unmount(SYNC_UNMOUNT, priv->data); ltfs_request_trace(FUSE_REQ_EXIT(REQ_UNMOUNT), 0, 0); diff --git a/src/utils/ltfsck.c b/src/utils/ltfsck.c index 31214a1a..007ae6ce 100644 --- a/src/utils/ltfsck.c +++ b/src/utils/ltfsck.c @@ -711,6 +711,7 @@ int check_ltfs_volume(struct ltfs_volume *vol, struct other_check_opts *opt) return LTFSCK_UNCORRECTED; } else { print_criteria_info(vol); + ltfs_set_commit_message_reason(SYNC_CHECK, vol); ltfs_unmount(SYNC_CHECK, vol); ltfsmsg(LTFS_INFO, 16022I); return LTFSCK_CORRECTED; @@ -1174,6 +1175,7 @@ int _rollback_dp(struct ltfs_volume *vol, struct other_check_opts *opt, struct t if (ret != LTFSCK_NO_ERRORS) ltfsmsg(LTFS_ERR, 16055E, ret); } else { + ltfs_set_commit_message_reason(SYNC_ROLLBACK, vol); ret = ltfs_write_index(ltfs_dp_id(vol), SYNC_ROLLBACK, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 16056E, ret); @@ -1288,6 +1290,7 @@ int rollback(struct ltfs_volume *vol, struct other_check_opts *opt) r.current_pos = ltfs_get_index_selfpointer(vol); ltfsmsg(LTFS_DEBUG, 16081D, ltfs_get_index_generation(vol), (int)r.current_pos.partition, (unsigned long long)r.current_pos.block); + ltfs_set_commit_message_reason(SYNC_ROLLBACK, vol); ltfs_unmount(SYNC_ROLLBACK, vol); vol->index = NULL; } From 8c8f6ccdb25ea91495d9b21cef1d0c0c080b994d Mon Sep 17 00:00:00 2001 From: Atsushi Abe Date: Fri, 21 Jun 2024 15:26:51 +0900 Subject: [PATCH 119/121] Introduce incremental index write down in XML format (#466) --- configure.ac | 32 +++ contrib/ut-incindex/README.md | 13 + contrib/ut-incindex/ut-basic.sh | 96 +++++++ contrib/ut-incindex/utils.sh | 197 ++++++++++++++ messages/internal_error/root.txt | 1 + messages/libltfs/root.txt | 8 + src/iosched/unified.c | 2 +- src/libltfs/arch/errormap.c | 1 + src/libltfs/inc_journal.c | 277 ++++++++++++++++++- src/libltfs/inc_journal.h | 34 ++- src/libltfs/ltfs.c | 229 +++++++++++----- src/libltfs/ltfs.h | 38 ++- src/libltfs/ltfs_error.h | 1 + src/libltfs/ltfs_fsops.c | 2 +- src/libltfs/ltfs_internal.c | 8 +- src/libltfs/periodic_sync.c | 3 +- src/libltfs/xattr.c | 33 ++- src/libltfs/xml_libltfs.h | 2 +- src/libltfs/xml_writer_libltfs.c | 449 ++++++++++++++++++++++++++++++- src/ltfs_fuse.c | 2 +- src/utils/ltfsck.c | 4 +- 21 files changed, 1320 insertions(+), 112 deletions(-) create mode 100644 contrib/ut-incindex/README.md create mode 100755 contrib/ut-incindex/ut-basic.sh create mode 100755 contrib/ut-incindex/utils.sh diff --git a/configure.ac b/configure.ac index b579b92f..369f66ae 100644 --- a/configure.ac +++ b/configure.ac @@ -184,6 +184,28 @@ AC_ARG_ENABLE([buggy_ifs], ) AC_MSG_RESULT([$buggy_ifs]) +dnl +dnl Handle --enable-xml-indent (default:no) +dnl +AC_MSG_CHECKING([whether to enable xml indentation for index]) +AC_ARG_ENABLE([xml_indent], + [AS_HELP_STRING([--enable-xml-indent],[Enable XML indentation for index])], + [xml_indent=$enableval], + [xml_indent=no] +) +AC_MSG_RESULT([$xml_indent]) + +dnl +dnl Handle --enable-format-spec25 (default:no) +dnl +AC_MSG_CHECKING([whether to enable format spec 2.5 support or not]) +AC_ARG_ENABLE([format_spec25], + [AS_HELP_STRING([--enable-format-spec25],[Support format spec 2.5])], + [format_spec25=$enableval], + [format_spec25=no] +) +AC_MSG_RESULT([$format_spec25]) + dnl dnl Handle extra compile flags for tape driver dnl @@ -499,6 +521,16 @@ then AM_CPPFLAGS="${AM_CPPFLAGS} -DPOSIXLINK_ONLY" fi +if test "x$xml_indent" = "xyes" +then + AM_CPPFLAGS="${AM_CPPFLAGS} -DINDENT_INDEXES" +fi + +if test "x$format_spec25" = "xyes" +then + AM_CPPFLAGS="${AM_CPPFLAGS} -DFORMAT_SPEC25" +fi + dnl dnl Specify CPU specific optimizer options for CRC calculation dnl diff --git a/contrib/ut-incindex/README.md b/contrib/ut-incindex/README.md new file mode 100644 index 00000000..ec3d9658 --- /dev/null +++ b/contrib/ut-incindex/README.md @@ -0,0 +1,13 @@ +# Unit Tests for incremental index + +This is a directory for having unit tests for incremental index feature. + +## How to run + +### Basic operation test + + 1. `cd` to this directory + 2. Run the basic test with `./ut-basic.sh [mount_point]` + - The test script formats a (filebackend) tape under `/tmp/ltfstape`, start ltfs and stop ltfs automatically. + - If `[mount_point]` is not specified, the script uses `/tmp/mnt` + - You can pass the specific `ltfs` binary directory with teh environmental value `LTFS_BIN_PATH` diff --git a/contrib/ut-incindex/ut-basic.sh b/contrib/ut-incindex/ut-basic.sh new file mode 100755 index 00000000..de61a29b --- /dev/null +++ b/contrib/ut-incindex/ut-basic.sh @@ -0,0 +1,96 @@ +#!/bin/sh + +source ./utils.sh + +MOUNTPOINT='/tmp/mnt' +TAPE_PATH='/tmp/ltfstape' + +if [ $# == 1 ]; then + MOUNTPOINT=$1 +elif [ $# -gt 2 ]; then + MOUNTPOINT=$1 + TAPE_PATH=$2 +fi + +# Format LTFS +FormatLTFS ${TAPE_PATH} +if [ $? != 0 ]; then + exit 1 +fi + +# Launch LTFS +LaunchLTFS ${MOUNTPOINT} ${TAPE_PATH} +if [ $? != 0 ]; then + exit 1 +fi + +# 1. CREATE DIRS +# Create a few dirs and files but all objects are handles by new dirs +echo "1. CREATE DIRS" +mkdir -p ${MOUNTPOINT}/dir1/dir11 +mkdir -p ${MOUNTPOINT}/dir1/dir12 +mkdir -p ${MOUNTPOINT}/dir2/dir21 +mkdir -p ${MOUNTPOINT}/dir2/dir22 +echo "AAA" > ${MOUNTPOINT}/dir1/file11 +echo "AAA" > ${MOUNTPOINT}/dir1/dir11/file111 +echo "AAA" > ${MOUNTPOINT}/dir1/dir11/file112 +IncrementalSync ${MOUNTPOINT} '1.CREATE_DIRS' +if [ $? != 0 ]; then + exit 1 +fi + +# 2. CREATE FILES +# Create files for checking file creation and directory traverse +echo "2. CREATE FILES" +echo "AAA" > ${MOUNTPOINT}/dir1/dir11/file113 +echo "AAA" > ${MOUNTPOINT}/dir1/dir12/file121 +echo "AAA" > ${MOUNTPOINT}/dir2/dir22/file221 +echo "AAA" > ${MOUNTPOINT}/dir2/file21 +IncrementalSync ${MOUNTPOINT} '2.CREATE_FILES' +if [ $? != 0 ]; then + exit 1 +fi + +# 3. MODIFY FILES +# Modify contents of files. Need to check /dir2 doesn't have meta-data on the incremental index +echo "3. MODIFY FILES" +echo "BBB" > ${MOUNTPOINT}/dir1/dir11/file111 +echo "BBB" > ${MOUNTPOINT}/dir2/dir22/file221 +echo "BBB" > ${MOUNTPOINT}/dir1/file11 +IncrementalSync ${MOUNTPOINT} '3.MODIFY_FILES' +if [ $? != 0 ]; then + exit 1 +fi + +# 4. MODIFY DIRS +# Modify directory's meta-data. Need to check both /dir1 and /dir1/dir11 has meta-data +# on the incremental index +echo "4. MODIFY DIRS" +AddXattr ${MOUNTPOINT}/dir1 'ut-attr1' 'val1' +echo "CCC" > ${MOUNTPOINT}/dir1/dir11/file111 +IncrementalSync ${MOUNTPOINT} '4.MODIFY_DIRS' +if [ $? != 0 ]; then + exit 1 +fi + +# 5. DELETE FILES +echo "5. DELETE FILES" +rm ${MOUNTPOINT}/dir1/dir11/* +IncrementalSync ${MOUNTPOINT} '5.DELETE_FILES' +if [ $? != 0 ]; then + exit 1 +fi + +# 6. DELETE DIR +echo "5. DELETE DIR" +rm -rf ${MOUNTPOINT}/dir1/dir11 +IncrementalSync ${MOUNTPOINT} '6.DELETE_DIR' +if [ $? != 0 ]; then + exit 1 +fi + +# Stop LTFS +StopLTFS ${MOUNTPOINT} ${TAPE_PATH} +if [ $? != 0 ]; then + exit 1 +fi diff --git a/contrib/ut-incindex/utils.sh b/contrib/ut-incindex/utils.sh new file mode 100755 index 00000000..ca79d3c6 --- /dev/null +++ b/contrib/ut-incindex/utils.sh @@ -0,0 +1,197 @@ +#!/bin/sh + +PLATFORM=`uname` +ECHO='/bin/echo' + +if [ "x${LTFS_BIN_PATH}" == 'x' ]; then + LTFS_BIN_PATH='/usr/local/bin' +fi + +AddXattr() +{ + if [ "x$1" == "x" ]; then + "Need to a target to set xattr" + return 1 + else + TARGET=$1 + fi + + if [ "x$2" == "x" ]; then + "Need to a name to set xattr" + return 1 + else + NAME=$2 + fi + + if [ "x$3" == "x" ]; then + "Need to a value to set xattr" + return 1 + else + VAL=$2 + fi + + ${ECHO} -n "Setting a xattr ${NAME} to ${TARGET} ... " + if [ "$PLATFORM" == "Darwin" ]; then + /usr/bin/xattr -w ${NAME} ${VAL} ${TARGET} + else + /usr/bin/attr -s ${NAME} -V ${VAL} ${TARGET} + fi + + if [ $? == 0 ]; then + ${ECHO} "Done" + return 0 + else + ${ECHO} "Failed" + return 1 + fi + +} + +IncrementalSync() +{ + if [ "x$1" == "x" ]; then + MOUNTPOINT='/tmp/mnt' + else + MOUNTPOINT=$1 + fi + + if [ "x$2" == "x" ]; then + MSG='Incremental Sync' + else + MSG=$2 + fi + + ${ECHO} -n "Syncing LTFS (Incremental) ... " + if [ "$PLATFORM" == "Darwin" ]; then + /usr/bin/xattr -w ltfs.vendor.IBM.IncrementalSync ${MSG} ${MOUNTPOINT} + else + /usr/bin/attr -s ltfs.vendor.IBM.IncrementalSync -V ${MSG} ${MOUNTPOINT} + fi + + if [ $? == 0 ]; then + ${ECHO} "Done" + return 0 + else + ${ECHO} "Failed" + return 1 + fi +} + +FullSync() +{ + if [ "x$1" == "x" ]; then + MOUNTPOINT='/tmp/mnt' + else + MOUNTPOINT=$1 + fi + + if [ "x$2" == "x" ]; then + MSG='Full Sync' + else + MSG=$2 + fi + + ${ECHO} "Syncing LTFS (Full) ... " + if [ "$PLATFORM" == "Darwin" ]; then + /usr/bin/xattr -w ltfs.vendor.IBM.FullSync ${MSG} ${MOUNTPOINT} + else + /usr/bin/attr -s ltfs.vendor.IBM.FullSync -V ${MSG} ${MOUNTPOINT} + fi + + if [ $? == 0 ]; then + ${ECHO} "Done" + return 0 + else + ${ECHO} "Failed" + return 1 + fi +} + +FormatLTFS() +{ + if [ "x$1" == "x" ]; then + TAPE_PATH='/tmp/ltfstape' + else + TAPE_PATH=$1 + fi + + if [ ! -d ${TAPE_PATH} ]; then + ${ECHO} "Creating tape directory for file backend: ${TAPE_PATH}" + mkdir -p ${TAPE_PATH} + if [ $? != 0 ]; then + ${ECHO} "Failed to create a tape path: ${TAPE_PATH}" + return 1 + fi + fi + + ${ECHO} "Formatting tape directory with the file backend on ${TAPE_PATH} ... " + ${LTFS_BIN_PATH}/mkltfs -f -e file -d ${TAPE_PATH} + if [ $? != 0 ]; then + ${ECHO} "Failed to format a tape path: ${TAPE_PATH}" + return 1 + fi + + ${ECHO} "Formatted the file backend on ${TAPE_PATH}" + return 0 +} + +LaunchLTFS() +{ + if [ "x$1" == "x" ]; then + MOUNTPOINT='/tmp/mnt' + else + MOUNTPOINT=$1 + fi + + if [ "x$2" == "x" ]; then + TAPE_PATH='/tmp/ltfstape' + else + TAPE_PATH=$2 + fi + + if [ ! -d ${MOUNTPOINT} ]; then + ${ECHO} "Creating mount point for LTFS: ${MOUNTPOINT}" + mkdir -p ${MOUNTPOINT} + if [ $? != 0 ]; then + ${ECHO} "Failed to create a mount point" + return 1 + fi + fi + + if [ ! -d ${TAPE_PATH} ]; then + ${ECHO} "Creating tape directory for file backend: ${TAPE_PATH}" + mkdir -p ${TAPE_PATH} + if [ $? != 0 ]; then + ${ECHO} "Failed to create a tape path: ${TAPE_PATH}" + return 1 + fi + + ${ECHO} "Formatting tape directory with the file backend" + ${LTFS_BIN_PATH}/mkltfs -f -e file -d ${TAPE_PATH} + if [ $? != 0 ]; then + ${ECHO} "Failed to format a tape path: ${TAPE_PATH}" + return 1 + fi + fi + + ${ECHO} "Launching LTFS with the file backend" + ${LTFS_BIN_PATH}/ltfs -o tape_backend=file -o sync_type=unmount -o devname=${TAPE_PATH} ${MOUNTPOINT} + if [ $? != 0 ]; then + ${ECHO} "Failed to launch LTFS on ${MOUNTPOINT}" + return 1 + fi + + ${ECHO} "LTFS is launched on ${MOUNTPOINT}" + return 0 +} + +StopLTFS() +{ + if [ "x$1" == "x" ]; then + MOUNTPOINT='/tmp/mnt' + else + MOUNTPOINT=$1 + fi + + sudo umount ${MOUNTPOINT} +} diff --git a/messages/internal_error/root.txt b/messages/internal_error/root.txt index 0df5c020..7692c8b9 100644 --- a/messages/internal_error/root.txt +++ b/messages/internal_error/root.txt @@ -309,6 +309,7 @@ root:table { I5048E:string{ "Unexpected partition map in a label." } I5049E:string{ "Unexpected blocksize in a label." } I5050E:string{ "Unexpected compression in a label." } + I5051E:string{ "Unsupported index type is specified." } // Special error codes I9997E:string{ "Child process error (ltfsck/mkltfs): %s (%d)." } diff --git a/messages/libltfs/root.txt b/messages/libltfs/root.txt index 938f8811..a4f0515b 100644 --- a/messages/libltfs/root.txt +++ b/messages/libltfs/root.txt @@ -836,6 +836,14 @@ v 17292I:string { "Current position is (%llu, %llu), Error position is (%llu, %llu)." } 17293E:string { "UUID in the index does not match the label." } + 17300I:string { "Wrote inc-index of %s (Gen = %lld, Part = %c, Pos = %lld, %s)." } + 17301I:string { "Info inc-index, Gen = %lld, Full Part = %c, Full Pos = %lld, Back Part = %c, Back Pos = %lld)." } + 17302E:string { "Path helper: Provided path must be an absolute path (%s)." } + 17303E:string { "Unexpected value was found in the reason of inc-journal entry (%d)." } + 17304E:string { "Unexpected value was provided to _xml_write_incremental_delete (%d)." } + 17305E:string { "Failed to construct a path helper (push: %d)." } + 17306E:string { "Failed to find a corresponded directory in path helper (push: %d)." } + // For Debug 19999I:string { "%s %s %d." } // DO NOT place messages with IDs 20000 or higher here! diff --git a/src/iosched/unified.c b/src/iosched/unified.c index af5265e7..a7ac70ca 100644 --- a/src/iosched/unified.c +++ b/src/iosched/unified.c @@ -2292,7 +2292,7 @@ int _unified_write_index_after_perm(int write_ret, struct unified_data *priv) } ltfs_set_commit_message_reason(SYNC_WRITE_PERM, priv->vol); - ret = ltfs_write_index(ltfs_ip_id(priv->vol), SYNC_WRITE_PERM, priv->vol); + ret = ltfs_write_index(ltfs_ip_id(priv->vol), SYNC_WRITE_PERM, LTFS_FULL_INDEX, priv->vol); return ret; } diff --git a/src/libltfs/arch/errormap.c b/src/libltfs/arch/errormap.c index 9c5f34b6..8459c23a 100644 --- a/src/libltfs/arch/errormap.c +++ b/src/libltfs/arch/errormap.c @@ -342,6 +342,7 @@ static struct error_map fuse_error_list[] = { { LTFS_XML_WRONG_PART_MAP, "I5048E", EINVAL}, { LTFS_XML_WRONG_BLOCKSIZE, "I5049E", EINVAL}, { LTFS_XML_WRONG_COMP, "I5050E", EINVAL}, + { LTFS_BAD_INDEX_TYPE, "I5051E", EINVAL}, { EDEV_NO_SENSE, "D0000E", EIO}, { EDEV_OVERRUN, "D0002E", EIO}, { EDEV_UNDERRUN, "D0003E", ENODATA}, diff --git a/src/libltfs/inc_journal.c b/src/libltfs/inc_journal.c index 67712f92..9c4386e0 100644 --- a/src/libltfs/inc_journal.c +++ b/src/libltfs/inc_journal.c @@ -57,22 +57,24 @@ static int _allocate_jentry(struct jentry **e, char *path, struct dentry* d) { struct jentry *ent = NULL; + *e = NULL; ent = calloc(1, sizeof(struct jentry)); if (!ent) { - ltfsmsg(LTFS_ERR, 11168E); + ltfsmsg(LTFS_ERR, 10001E, "allocating a jentry"); return -LTFS_NO_MEMORY; } ent->id.full_path = path; ent->id.uid = d->uid; + *e = ent; return 0; } -static int _dispose_jentry(struct jentry *ent) +static inline int _dispose_jentry(struct jentry *ent) { if (ent) { if (ent->id.full_path) @@ -115,7 +117,7 @@ int incj_create(char *ppath, struct dentry *d, struct ltfs_volume *vol) /* Create full path of created object and jentry */ len = asprintf(&full_path, "%s/%s", ppath, d->name.name); if (len < 0) { - ltfsmsg(LTFS_ERR, 11168E); + ltfsmsg(LTFS_ERR, 10001E, "full path of a jentry"); vol->journal_err = true; return -LTFS_NO_MEMORY; } @@ -135,7 +137,7 @@ int incj_create(char *ppath, struct dentry *d, struct ltfs_volume *vol) if (d->isdir) { jdir = calloc(1, sizeof(struct jcreated_entry)); if (!jdir) { - ltfsmsg(LTFS_ERR, 11168E); + ltfsmsg(LTFS_ERR, 10001E, "allocating a jcreated_entry"); return -LTFS_NO_MEMORY; } @@ -169,7 +171,7 @@ int incj_modify(char *path, struct dentry *d, struct ltfs_volume *vol) } /* Skip journal modification because it is already existed */ - HASH_FIND(hh, vol->journal, &ent->id, sizeof(struct jentry), ent); + HASH_FIND(hh, vol->journal, &d->uid, sizeof(struct jentry), ent); if (ent) { return 0; } @@ -247,10 +249,10 @@ int incj_rmfile(char *path, struct dentry *d, struct ltfs_volume *vol) } } - /* Create full path of created object and jentry */ + /* Create full path of deleted object and jentry */ full_path = strdup(path); if (!full_path) { - ltfsmsg(LTFS_ERR, 11168E); + ltfsmsg(LTFS_ERR, 10001E, "duplicating a path for deleted file"); vol->journal_err = true; return -LTFS_NO_MEMORY; } @@ -262,6 +264,13 @@ int incj_rmfile(char *path, struct dentry *d, struct ltfs_volume *vol) } ent->reason = DELETE_FILE; + ent->name.percent_encode = d->name.percent_encode; + ent->name.name = strdup(d->name.name); + if (!ent->name.name) { + ltfsmsg(LTFS_ERR, 10001E, "duplicating a name of deleted file"); + vol->journal_err = true; + return -LTFS_NO_MEMORY; + } HASH_ADD(hh, vol->journal, id, sizeof(struct jentry), ent); @@ -320,7 +329,7 @@ int incj_rmdir(char *path, struct dentry *d, struct ltfs_volume *vol) /* Create full path of created object and jentry */ full_path = strdup(path); if (!full_path) { - ltfsmsg(LTFS_ERR, 11168E); + ltfsmsg(LTFS_ERR, 10001E, "duplicating a path of deleted directory"); vol->journal_err = true; return -LTFS_NO_MEMORY; } @@ -332,12 +341,24 @@ int incj_rmdir(char *path, struct dentry *d, struct ltfs_volume *vol) } ent->reason = DELETE_DIRECTORY; + ent->name.percent_encode = d->name.percent_encode; + ent->name.name = strdup(d->name.name); + if (!ent->name.name) { + ltfsmsg(LTFS_ERR, 10001E, "duplicating a name of deleted directory"); + vol->journal_err = true; + return -LTFS_NO_MEMORY; + } HASH_ADD(hh, vol->journal, id, sizeof(struct jentry), ent); return 0; } +int incj_dispose_jentry(struct jentry *ent) +{ + return (_dispose_jentry(ent)); +} + /** * Clear all entries into the incremental journal */ @@ -394,6 +415,11 @@ static inline int dig_path(char *p, struct ltfs_index *idx) return ret; } +void incj_sort(struct ltfs_volume *vol) +{ + HASH_SORT(vol->journal, _by_path); +} + /** * This is a function for debug. Print contents of the journal and the created * directory list to stdout. @@ -412,7 +438,7 @@ void incj_dump(struct ltfs_volume *vol) } printf("--------------------------------------------------------------------------------\n"); - HASH_SORT(vol->journal, _by_path); + incj_sort(vol); HASH_ITER(hh, vol->journal, ent, tmp) { printf("JOURNAL: %s, %llu, %s, ", ent->id.full_path, (unsigned long long)ent->id.uid, reason[ent->reason]); if (!ent->dentry) @@ -425,7 +451,7 @@ void incj_dump(struct ltfs_volume *vol) } else printf("file\n"); - parent= strdup(ent->id.full_path); + parent = strdup(ent->id.full_path); fs_split_path(parent, &filename, strlen(parent) + 1); if (prev_parent) { @@ -448,3 +474,234 @@ void incj_dump(struct ltfs_volume *vol) return; } + +int incj_create_path_helper(const char *dpath, struct incj_path_helper **pm, struct ltfs_volume *vol) +{ + struct incj_path_helper *ipm; + char *wp = NULL, *tmp = NULL, *dname = NULL; + int ret = 0; + + *pm = NULL; + + ipm = calloc(1, sizeof(struct incj_path_helper)); + if (!ipm) { + ltfsmsg(LTFS_ERR, 10001E, "allocating a path helper"); + return -LTFS_NO_MEMORY; + } + + if (dpath[0] != '/') { + /* Provided path must be a absolute path */ + ltfsmsg(LTFS_ERR, 17302E, dpath); + free(ipm); + return -LTFS_INVALID_PATH; + } + + ipm->vol = vol; + + if (strcmp(dpath, "/") == 0) { + /* Provided path is the root, return good */ + *pm = ipm; + return 0; + } + + wp = strdup(dpath); + if (!wp) { + ltfsmsg(LTFS_ERR, 10001E, "duplicating a directory path for path helper"); + free(ipm); + return -LTFS_NO_MEMORY; + } + + for (dname = strtok_r(wp, "/", &tmp); dname != NULL; dname = strtok_r(NULL, "/", &tmp)) { + ret = incj_push_directory(dname, ipm); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17305E, ret); + free(wp); + incj_destroy_path_helper(ipm); + return ret; + } + } + + free(wp); + *pm = ipm; + + return 0; +} + +int incj_destroy_path_helper(struct incj_path_helper *pm) +{ + struct incj_path_element *cur, *next; + + cur = pm->head; + + while (cur) { + next = cur->next; + if (cur->d) + fs_release_dentry(cur->d); + if (cur->name) + free(cur->name); + free(cur); + cur = next; + } + + free(pm); + return 0; +} + +int incj_push_directory(char *name, struct incj_path_helper *pm) +{ + int ret = 0; + struct incj_path_element *ipelm = NULL, *cur_tail = NULL; + struct dentry *parent = NULL; + + ipelm = calloc(1, sizeof(struct incj_path_element)); + if (!ipelm) { + ltfsmsg(LTFS_ERR, 10001E, "allocating a path element on push"); + return -LTFS_NO_MEMORY; + } + + /* Set name field of new path element */ + ipelm->name = strdup(name); + if (!ipelm->name) { + ltfsmsg(LTFS_ERR, 10001E, "duplicating a path of pushing directory"); + incj_destroy_path_helper(pm); + return -LTFS_NO_MEMORY; + } + + /* Set dentry field of new path element */ + if (pm->elems) + parent = pm->tail->d; + else + parent = pm->vol->index->root; + + ret = fs_directory_lookup(parent, name, &ipelm->d); + if (ret) { + ltfsmsg(LTFS_ERR, 17306E, ret); + free(ipelm->name); + free(ipelm); + incj_destroy_path_helper(pm); + return -LTFS_INVALID_PATH; + } + + /* Modify path chain and # of elements */ + if (!pm->elems) { + pm->head = ipelm; + pm->tail = ipelm; + } else { + cur_tail = pm->tail; + cur_tail->next = ipelm; + ipelm->prev = cur_tail; + pm->tail = ipelm; + } + + pm->elems++; + + return 0; +} + +int incj_pop_directory(struct incj_path_helper *pm) +{ + struct incj_path_element *cur_tail = NULL, *new_tail = NULL; + + if (!pm->elems) { + /* Must have one or more elements */ + return -LTFS_UNEXPECTED_VALUE; + } + + cur_tail = pm->tail; + new_tail = cur_tail->prev; + + new_tail->next = NULL; + pm->tail = new_tail; + + pm->elems--; + if (!pm->elems) { + pm->head = NULL; + } + + if (cur_tail->d) + fs_release_dentry(cur_tail->d); + if (cur_tail->name) + free(cur_tail->name); + free(cur_tail); + + return 0; +} + +int incj_compare_path(struct incj_path_helper *now, struct incj_path_helper *next, + int *matches, int *pops, bool *perfect_match) +{ + int ret = 0, matched = 0; + struct incj_path_element *cur1 = NULL, *cur2 = NULL; + + *matches = 0; + *pops = 0; + *perfect_match = false; + + cur1 = now->head; + cur2 = next->head; + + if (!cur1 && !cur2) { + /* Both are root */ + *perfect_match = true; + return 0; + } + + while (cur1 && cur2) { + if (cur1->d != cur2->d) + break; + matched++; + cur1 = cur1->next; + cur2 = cur2->next; + } + + *matches = matched; + *pops = now->elems - *matches; + + if (!cur1 && !cur2) + *perfect_match = true; + + return ret; +} + +char* incj_get_path(struct incj_path_helper *pm) +{ + char *path = NULL, *path_old = NULL; + struct incj_path_element *cur = NULL; + int ret = 0; + + cur = pm->head; + + if (!cur) { + /* Root directory */ + ret = asprintf(&path, "/"); + if (ret < 0) { + /* memory allocation error */ + return NULL; + } + return path; + } + + while (cur) { + if (path) path_old = path; + ret = asprintf(&path, "%s/%s", path_old, cur->name); + if (ret < 0) { + /* memory allocation error */ + free(path_old); + return NULL; + } + free(path_old); + + cur++; + } + + if (path) path_old = path; + ret = asprintf(&path, "/%s", path_old); + if (ret < 0) { + /* memory allocation error */ + free(path_old); + return NULL; + } + free(path_old); + + return path; +} diff --git a/src/libltfs/inc_journal.h b/src/libltfs/inc_journal.h index 1b4b1b2f..14784857 100644 --- a/src/libltfs/inc_journal.h +++ b/src/libltfs/inc_journal.h @@ -81,9 +81,10 @@ struct journal_id { * Journal entry */ struct jentry { - struct journal_id id; /**< ID of the journal entry (key of the hash table) */ - enum journal_reason reason; /**< Reason of the entry */ - struct dentry *dentry; /**< Target dentry if required */ + struct journal_id id; /**< ID of the journal entry (key of the hash table) */ + enum journal_reason reason; /**< Reason of the entry */ + struct dentry *dentry; /**< Target dentry if required */ + struct ltfs_name name; /**< Name of entry for delete */ UT_hash_handle hh; }; @@ -95,13 +96,40 @@ struct jcreated_entry { char *path; }; +/** + * + */ +struct incj_path_element { + struct incj_path_element *prev; + struct incj_path_element *next; + char* name; + struct dentry *d; +}; + +struct incj_path_helper { + struct incj_path_element *head; + struct incj_path_element *tail; + struct ltfs_volume *vol; + unsigned int elems; +}; + int incj_create(char *ppath, struct dentry *d, struct ltfs_volume *vol); int incj_modify(char *path, struct dentry *d, struct ltfs_volume *vol); int incj_rmfile(char *path, struct dentry *d, struct ltfs_volume *vol); int incj_rmdir(char *path, struct dentry *d, struct ltfs_volume *vol); +int incj_dispose_jentry(struct jentry *ent); int incj_clear(struct ltfs_volume *vol); +void incj_sort(struct ltfs_volume *vol); void incj_dump(struct ltfs_volume *vol); +int incj_create_path_helper(const char *path, struct incj_path_helper **pm, struct ltfs_volume *vol); +int incj_destroy_path_helper(struct incj_path_helper *pm); +int incj_push_directory(char *name, struct incj_path_helper *pm); +int incj_pop_directory(struct incj_path_helper *pm); +int incj_compare_path(struct incj_path_helper *p1, struct incj_path_helper *p2, + int *matches, int *pops, bool *perfect_match); +char* incj_get_path(struct incj_path_helper *pm); + #ifdef __cplusplus } #endif diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index 40624c6e..baa7aae3 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1780,7 +1780,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec INTERRUPTED_GOTO(ret, out_unlock); ltfsmsg(LTFS_INFO, 11022I); ltfs_set_commit_message_reason(SYNC_RECOVERY, vol); - ret = ltfs_write_index(vol->label->partid_ip, SYNC_RECOVERY, vol); + ret = ltfs_write_index(vol->label->partid_ip, SYNC_RECOVERY, LTFS_FULL_INDEX, vol); if (ret < 0) goto out_unlock; } @@ -2073,7 +2073,7 @@ int ltfs_unmount(char *reason, struct ltfs_volume *vol) if (vol->mount_type == MOUNT_NORMAL && (ltfs_is_dirty(vol) || vol->index->selfptr.partition != ltfs_ip_id(vol)) && (vollock != PWE_MAM_IP && vollock != PWE_MAM_BOTH)) { - ret = ltfs_write_index(ltfs_ip_id(vol), reason, vol); + ret = ltfs_write_index(ltfs_ip_id(vol), reason, LTFS_FULL_INDEX, vol); if (NEED_REVAL(ret)) { ret = ltfs_revalidate(true, vol); if (ret == 0) { @@ -2435,13 +2435,15 @@ size_t ltfs_max_cache_size(struct ltfs_volume *vol) * when the cartridge is known to be in a sane state. * The caller must hold vol->lock for write if thread safety is required. * @param partition partition in which the schema should be written to + * @param reason the reason to write down an index + * @param type type of index to write (shall be LTFS_FULL_INDEX or LTFS_INCREMENTAL_INDEX) * @param vol LTFS volume * @return 0 on success or a negative value on error */ -int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) +int ltfs_write_index(char partition, char *reason, enum ltfs_index_type type, struct ltfs_volume *vol) { int ret, ret_mam; - struct tape_offset old_selfptr, old_backptr; + struct tape_offset old_selfptr, old_backptr, old_selfptr_inc, old_backptr_inc; struct ltfs_timespec modtime_old = { .tv_sec = 0, .tv_nsec = 0 }; bool generation_inc = false; struct tc_position physical_selfptr; @@ -2459,7 +2461,30 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) return ret; } - incj_clear(vol); /* Clear incremental journal data */ + if (type == LTFS_INDEX_AUTO) { + if (vol->index->full_index_interval) { + if (vol->index->full_index_to_go) + type = LTFS_INCREMENTAL_INDEX; + else + type = LTFS_FULL_INDEX; + } else { + type = LTFS_FULL_INDEX; + } + } + + switch (type) { + case LTFS_FULL_INDEX: + if (vol->index->full_index_interval) + vol->index->full_index_to_go = vol->index->full_index_interval; + break; + case LTFS_INCREMENTAL_INDEX: + if (vol->index->full_index_interval && vol->index->full_index_to_go) + vol->index->full_index_to_go--; + break; + default: + /* TODO: Unexpected index type error */ + break; + } bc_print = _get_barcode(vol); @@ -2497,7 +2522,10 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) /* Surpress on-disk index cache write on the recursive call */ cache_path_save = vol->index_cache_path_w; vol->index_cache_path_w = NULL; - ret = ltfs_write_index(ltfs_dp_id(vol), reason, vol); + + type = LTFS_FULL_INDEX; + incj_clear(vol); /* Clear incremental journal data */ + ret = ltfs_write_index(ltfs_dp_id(vol), reason, type, vol); /* Restore cache path to handle on-disk index cache */ vol->index_cache_path_w = cache_path_save; @@ -2536,10 +2564,10 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) /* update index generation */ if (ltfs_is_dirty(vol)) { - modtime_old = vol->index->mod_time; generation_inc = true; - get_current_timespec(&vol->index->mod_time); ++vol->index->generation; + modtime_old = vol->index->mod_time; + get_current_timespec(&vol->index->mod_time); } /* locate to append position */ @@ -2554,38 +2582,61 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) } /* update back pointer */ - old_backptr = vol->index->backptr; - if (vol->index->selfptr.partition == ltfs_dp_id(vol)) - memcpy(&vol->index->backptr, &vol->index->selfptr, sizeof(struct tape_offset)); + if (type == LTFS_FULL_INDEX) { + old_backptr = vol->index->backptr; + if (vol->index->selfptr.partition == ltfs_dp_id(vol)) + memcpy(&vol->index->backptr, &vol->index->selfptr, sizeof(struct tape_offset)); + } else { + old_backptr_inc = vol->index->backptr_inc; + memcpy(&vol->index->backptr_inc, &vol->index->selfptr_inc, sizeof(struct tape_offset)); + } /* update self pointer */ ret = tape_get_position(vol->device, &physical_selfptr); if (ret < 0) { ltfsmsg(LTFS_ERR, 11081E, ret); - if (generation_inc) { - vol->index->mod_time = modtime_old; - --vol->index->generation; + if (type == LTFS_FULL_INDEX) { + if (generation_inc) { + vol->index->mod_time = modtime_old; + --vol->index->generation; + } + vol->index->backptr = old_backptr; + } else { + vol->index->backptr_inc = old_backptr_inc; } - vol->index->backptr = old_backptr; goto out_write_perm; } - old_selfptr = vol->index->selfptr; - vol->index->selfptr.partition = partition; - vol->index->selfptr.partition = vol->label->part_num2id[physical_selfptr.partition]; - vol->index->selfptr.block = physical_selfptr.block; - ++vol->index->selfptr.block; /* point to first data block, not preceding filemark */ + + if (type == LTFS_FULL_INDEX) { + old_selfptr = vol->index->selfptr; + vol->index->selfptr.partition = partition; + vol->index->selfptr.partition = vol->label->part_num2id[physical_selfptr.partition]; + vol->index->selfptr.block = physical_selfptr.block; + ++vol->index->selfptr.block; /* point to first data block, not preceding filemark */ + } else { + old_selfptr_inc = vol->index->selfptr_inc; + vol->index->selfptr_inc.partition = partition; + vol->index->selfptr_inc.partition = vol->label->part_num2id[physical_selfptr.partition]; + vol->index->selfptr_inc.block = physical_selfptr.block; + ++vol->index->selfptr_inc.block; /* point to first data block, not preceding filemark */ + } /* Write the Index. */ if ((partition == ltfs_ip_id(vol)) && !vol->ip_index_file_end) { ret = tape_write_filemark(vol->device, 0, true, true, false); // Flush data before writing FM if (ret < 0) { ltfsmsg(LTFS_ERR, 11326E, ret); - if (generation_inc) { - vol->index->mod_time = modtime_old; - --vol->index->generation; + if (type == LTFS_FULL_INDEX) { + if (generation_inc) { + vol->index->mod_time = modtime_old; + --vol->index->generation; + } + vol->index->backptr = old_backptr; + vol->index->selfptr = old_selfptr; + } else { + vol->index->backptr_inc = old_backptr_inc; + vol->index->selfptr_inc = old_selfptr_inc; } - vol->index->backptr = old_backptr; - vol->index->selfptr = old_selfptr; if (IS_WRITE_PERM(-ret)) update_vollock = true; @@ -2600,12 +2651,18 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) ret = tape_write_filemark(vol->device, 1, true, true, true); // immediate WFM if (ret < 0) { ltfsmsg(LTFS_ERR, 11082E, ret); - if (generation_inc) { - vol->index->mod_time = modtime_old; - --vol->index->generation; + + if (type == LTFS_FULL_INDEX) { + if (generation_inc) { + vol->index->mod_time = modtime_old; + --vol->index->generation; + } + vol->index->backptr = old_backptr; + vol->index->selfptr = old_selfptr; + } else { + vol->index->backptr_inc = old_backptr_inc; + vol->index->selfptr_inc = old_selfptr_inc; } - vol->index->backptr = old_backptr; - vol->index->selfptr = old_selfptr; if (IS_WRITE_PERM(-ret)) update_vollock = true; @@ -2614,15 +2671,21 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) } /* Actually write index to tape and disk if vol->index_cache_path is existed */ - ret = xml_schema_to_tape(reason, vol); + ret = xml_schema_to_tape(reason, type, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 11083E, ret); - if (generation_inc) { - vol->index->mod_time = modtime_old; - --vol->index->generation; + + if (type == LTFS_FULL_INDEX) { + if (generation_inc) { + vol->index->mod_time = modtime_old; + --vol->index->generation; + } + vol->index->backptr = old_backptr; + vol->index->selfptr = old_selfptr; + } else { + vol->index->backptr_inc = old_backptr_inc; + vol->index->selfptr_inc = old_selfptr_inc; } - vol->index->backptr = old_backptr; - vol->index->selfptr = old_selfptr; if (IS_WRITE_PERM(-ret)) update_vollock = true; @@ -2630,38 +2693,60 @@ int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol) goto out_write_perm; } - /* Update MAM parameters. */ - if (partition == ltfs_ip_id(vol)) - vol->ip_index_file_end = true; - else /* partition == ltfs_dp_id(vol) */ - vol->dp_index_file_end = true; + if (type == LTFS_FULL_INDEX) { + /* Update MAM parameters. */ + if (partition == ltfs_ip_id(vol)) + vol->ip_index_file_end = true; + else /* partition == ltfs_dp_id(vol) */ + vol->dp_index_file_end = true; + + /* The MAM may be inaccessible, or it may not be available on this medium. Either way, + * ignore failures when updating MAM parameters. */ + ltfs_update_cart_coherency(vol); + + ltfsmsg(LTFS_INFO, 17236I, + bc_print, + (unsigned long long)vol->index->generation, + vol->index->selfptr.partition, + (unsigned long long)vol->index->selfptr.block, + tape_get_serialnumber(vol->device)); + + /* update append position */ + if (partition == ltfs_ip_id(vol)) { + tape_set_ip_append_position(vol->device, ltfs_part_id2num(partition, vol), + vol->index->selfptr.block - 1); + } - /* The MAM may be inaccessible, or it may not be available on this medium. Either way, - * ignore failures when updating MAM parameters. */ - ltfs_update_cart_coherency(vol); + if (dcache_initialized(vol)) { + dcache_set_dirty(false, vol); + if (generation_inc) { + dcache_set_generation(vol->index->generation, vol); + } + } - ltfsmsg(LTFS_INFO, 17236I, - bc_print, - (unsigned long long)vol->index->generation, - vol->index->selfptr.partition, - (unsigned long long)vol->index->selfptr.block, - tape_get_serialnumber(vol->device)); + /* Clear incremental index metrix */ + vol->index->backptr_inc.partition = 0; + vol->index->backptr_inc.block = 0; + vol->index->selfptr_inc.partition = 0; + vol->index->selfptr_inc.block = 0; - /* update append position */ - if (partition == ltfs_ip_id(vol)) { - tape_set_ip_append_position(vol->device, ltfs_part_id2num(partition, vol), - vol->index->selfptr.block - 1); - } - - if (dcache_initialized(vol)) { - dcache_set_dirty(false, vol); - if (generation_inc) { - dcache_set_generation(vol->index->generation, vol); - } + incj_clear(vol); /* Clear incremental journal data */ + ltfs_unset_index_dirty(true, vol->index); + } else { + ltfsmsg(LTFS_INFO, 17300I, + bc_print, + (unsigned long long)vol->index->generation, + vol->index->selfptr_inc.partition, + (unsigned long long)vol->index->selfptr_inc.block, + tape_get_serialnumber(vol->device)); + ltfsmsg(LTFS_INFO, 17301I, + (unsigned long long)vol->index->generation, + vol->index->backptr.partition, + (unsigned long long)vol->index->backptr.block, + vol->index->backptr_inc.partition, + (unsigned long long)vol->index->backptr_inc.block); } - ltfs_unset_index_dirty(true, vol->index); - out_write_perm: if (write_perm) { /* Restore device->write_error flag that is disabled to handle a write perm on DP */ @@ -3133,7 +3218,7 @@ int ltfs_format_tape(struct ltfs_volume *vol, int density_code, bool destructive return ret; ltfsmsg(LTFS_INFO, 11278I, vol->label->partid_dp); /* "Writing Index to ..." */ ltfs_set_commit_message_reason(SYNC_FORMAT, vol); - ret = ltfs_write_index(vol->label->partid_dp, SYNC_FORMAT, vol); + ret = ltfs_write_index(vol->label->partid_dp, SYNC_FORMAT, LTFS_FULL_INDEX, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 11279E, vol->label->partid_dp, ret); return ret; @@ -3146,7 +3231,7 @@ int ltfs_format_tape(struct ltfs_volume *vol, int density_code, bool destructive if (ret < 0) return ret; ltfsmsg(LTFS_INFO, 11278I, vol->label->partid_ip); /* "Writing Index to ..." */ - ret = ltfs_write_index(vol->label->partid_ip, SYNC_FORMAT, vol); + ret = ltfs_write_index(vol->label->partid_ip, SYNC_FORMAT, LTFS_FULL_INDEX, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 11279E, vol->label->partid_ip, ret); return ret; @@ -3547,11 +3632,13 @@ int ltfs_revalidate(bool have_write_lock, struct ltfs_volume *vol) /** * Write index to tape if the index is dirty, and if there is space available * on the data partition. - * @param vol LTFS volume + * @param reason the reason to write an index * @param index_locking Take index lock while writing an index + * @param type index type to write + * @param vol LTFS volume * @return 0 on success or a negative value on error */ -int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) +int ltfs_sync_index(char *reason, bool index_locking, enum ltfs_index_type type, struct ltfs_volume *vol) { int ret = 0, ret_r = 0; bool dirty; @@ -3559,6 +3646,10 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) bool dp_index_file_end, ip_index_file_end; char *bc_print = NULL; +#ifndef FORMAT_SPEC25 + type = LTFS_FULL_INDEX; +#endif + start: ret = ltfs_get_partition_readonly(ltfs_dp_id(vol), vol); if (ret < 0 && ret != -LTFS_LESS_SPACE) @@ -3613,7 +3704,7 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) releasewrite_mrsw(&vol->lock); return ret; } - ret = ltfs_write_index(partition, reason, vol); + ret = ltfs_write_index(partition, reason, type, vol); if (IS_WRITE_PERM(-ret) && partition == ltfs_dp_id(vol)) { /* * TODO: Need to determine the last record on DP of the tape and cleanup @@ -3630,7 +3721,7 @@ int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol) * request. */ ltfs_set_commit_message_reason(SYNC_WRITE_PERM, vol); - ret_r = ltfs_write_index(ltfs_ip_id(vol), SYNC_WRITE_PERM, vol); + ret_r = ltfs_write_index(ltfs_ip_id(vol), SYNC_WRITE_PERM, LTFS_FULL_INDEX, vol); if (!ret_r) { ltfsmsg(LTFS_INFO, 11344I, bc_print); ret = -LTFS_SYNC_FAIL_ON_DP; diff --git a/src/libltfs/ltfs.h b/src/libltfs/ltfs.h index 18f4b2dd..ec518f1a 100644 --- a/src/libltfs/ltfs.h +++ b/src/libltfs/ltfs.h @@ -154,16 +154,34 @@ struct device_data; #define LTFS_LABEL_VERSION_MIN MAKE_LTFS_VERSION(1,0,0) /* Min supported label version */ #define LTFS_LABEL_VERSION_MAX MAKE_LTFS_VERSION(2,99,99) /* Max supported label version */ + +#ifdef FORMAT_SPEC25 +#define LTFS_LABEL_VERSION MAKE_LTFS_VERSION(2,5,0) /* Written label version */ +#define LTFS_LABEL_VERSION_STR "2.5.0" /* Label version string */ +#else #define LTFS_LABEL_VERSION MAKE_LTFS_VERSION(2,4,0) /* Written label version */ #define LTFS_LABEL_VERSION_STR "2.4.0" /* Label version string */ +#endif #define LTFS_INDEX_VERSION_MIN MAKE_LTFS_VERSION(1,0,0) /* Min supported index version */ #define LTFS_INDEX_VERSION_MAX MAKE_LTFS_VERSION(2,99,99) /* Max supported index version */ + +#ifdef FORMAT_SPEC25 +#define LTFS_INDEX_VERSION MAKE_LTFS_VERSION(2,5,0) /* Written index version */ +#define LTFS_INDEX_VERSION_STR "2.5.0" /* Index version string */ +#else #define LTFS_INDEX_VERSION MAKE_LTFS_VERSION(2,4,0) /* Written index version */ #define LTFS_INDEX_VERSION_STR "2.4.0" /* Index version string */ +#endif #define INDEX_MAX_COMMENT_LEN 65536 /* Maximum comment field length (per LTFS Format) */ +enum ltfs_index_type { + LTFS_INDEX_AUTO = 0, /**< Select index type to write based on specified options */ + LTFS_FULL_INDEX, /**< Forcibly write full index */ + LTFS_INCREMENTAL_INDEX /**< Forcibly write incremental index */ +}; + #define LTFS_NO_BARCODE "NO_BARCODE" #ifndef __APPLE_MAKEFILE__ @@ -363,7 +381,7 @@ typedef enum mam_advisory_lock_status { PWE_MAM_IP = 5, /* Single write perm on IP (Set VOL_IP_PERM__ERR) */ PWE_MAM_BOTH = 6, /* Double write perm (Set both VOL_DP_PERM_ERR and VOL_IP_PERM_ERR) */ NOLOCK_MAM = 128, /* From HPE */ -} mam_lockval; +} mam_lockval_t; #define IS_SINGLE_WRITE_PERM(stat) (stat == PWE_MAM || (stat == PWE_MAM_DP || stat == PWE_MAM_IP) ) #define IS_DOUBLE_WRITE_PERM(stat) (stat == PWE_MAM_BOTH) @@ -456,7 +474,7 @@ struct ltfs_volume { char *mountpoint; /**< Store mount point for Live Link (SDE) */ size_t mountpoint_len; /**< Store mount point path length (SDE) */ struct tape_attr *t_attr; /**< Tape Attribute data */ - mam_lockval lock_status; /**< Total volume lock status from t_attr->vollock and index->vollock */ + mam_lockval_t lock_status; /**< Total volume lock status from t_attr->vollock and index->vollock */ struct ltfs_timespec first_locate; /**< Time to first locate */ int file_open_count; /**< Number of opened files */ @@ -520,10 +538,10 @@ struct ltfs_index { struct index_criteria index_criteria; /**< Active index criteria */ struct dentry *root; /**< The directory tree */ - ltfs_mutex_t rename_lock; /**< Controls name tree access during renames */ + ltfs_mutex_t rename_lock; /**< Controls name tree access during renames */ /* Update tracking */ - ltfs_mutex_t dirty_lock; /**< Controls access to the update tracking bits */ + ltfs_mutex_t dirty_lock; /**< Controls access to the update tracking bits */ bool dirty; /**< Set on metadata update, cleared on write to tape */ bool atime_dirty; /**< Set on atime update, cleared on write to tape */ bool use_atime; /**< Set if atime updates should make the index dirty */ @@ -542,7 +560,13 @@ struct ltfs_index { size_t symerr_count; /**< Number of conflicted symlink dentries */ struct dentry **symlink_conflict; /**< symlink/extent conflicted dentries */ - mam_lockval vollock; /**< volume lock status on index */ + mam_lockval_t vollock; /**< volume lock status on index */ + + /* Incremental index */ + uint64_t full_index_interval; /**< Number of indexes between full indexes */ + uint64_t full_index_to_go; /**< How many incremental index shall be written to the next full index */ + struct tape_offset selfptr_inc; /**< self-pointer of previous incremental index (to prior generation on data partition) */ + struct tape_offset backptr_inc; /**< back pointer of previous incremental index (to prior generation on data partition) */ }; struct ltfs_direntry { @@ -697,14 +721,14 @@ int ltfs_parse_library_backend_opts(void *opt_args, void *opts); void ltfs_set_index_dirty(bool locking, bool atime, struct ltfs_index *idx); void ltfs_unset_index_dirty(bool update_version, struct ltfs_index *idx); -int ltfs_write_index(char partition, char *reason, struct ltfs_volume *vol); +int ltfs_write_index(char partition, char *reason, enum ltfs_index_type type, struct ltfs_volume *vol); int ltfs_save_index_to_disk(const char *work_dir, char *reason, char id, struct ltfs_volume *vol); char ltfs_dp_id(struct ltfs_volume *vol); char ltfs_ip_id(struct ltfs_volume *vol); const char *ltfs_get_volume_uuid(struct ltfs_volume *vol); -int ltfs_sync_index(char *reason, bool index_locking, struct ltfs_volume *vol); +int ltfs_sync_index(char *reason, bool index_locking, enum ltfs_index_type type, struct ltfs_volume *vol); int ltfs_traverse_index_forward(struct ltfs_volume *vol, char partition, unsigned int gen, bool skip_dir, f_index_found func, void **list, void *priv); diff --git a/src/libltfs/ltfs_error.h b/src/libltfs/ltfs_error.h index ab661262..703b6324 100644 --- a/src/libltfs/ltfs_error.h +++ b/src/libltfs/ltfs_error.h @@ -317,6 +317,7 @@ #define LTFS_XML_WRONG_PART_MAP 5048 /* Unexpected partition map in a label */ #define LTFS_XML_WRONG_BLOCKSIZE 5049 /* Unexpected blocksize in a label */ #define LTFS_XML_WRONG_COMP 5050 /* Unexpected compression in a label */ +#define LTFS_BAD_INDEX_TYPE 5051 /* Unsupported index type is specified */ #define LTFS_ERR_MAX 19999 diff --git a/src/libltfs/ltfs_fsops.c b/src/libltfs/ltfs_fsops.c index be046720..98868d41 100644 --- a/src/libltfs/ltfs_fsops.c +++ b/src/libltfs/ltfs_fsops.c @@ -2110,7 +2110,7 @@ int ltfs_fsops_volume_sync(char *reason, struct ltfs_volume *vol) ltfs_set_commit_message_reason_unlocked(reason, vol); ltfs_mutex_unlock(&vol->index->dirty_lock); - ret = ltfs_sync_index(reason, true, vol); + ret = ltfs_sync_index(reason, true, LTFS_INDEX_AUTO, vol); return ret; } diff --git a/src/libltfs/ltfs_internal.c b/src/libltfs/ltfs_internal.c index cbd78c74..028e32bf 100644 --- a/src/libltfs/ltfs_internal.c +++ b/src/libltfs/ltfs_internal.c @@ -1326,11 +1326,11 @@ int ltfs_check_medium(bool fix, bool deep, bool recover_extra, bool recover_syml ltfs_set_commit_message_reason(SYNC_RECOVERY, vol); if (! dp_have_index || dp_blocks_after) { ltfsmsg(LTFS_INFO, 17259I, "DP", vol->index->selfptr.partition, (unsigned long long)vol->index->selfptr.block); - ret = ltfs_write_index(vol->label->partid_dp, SYNC_RECOVERY, vol); + ret = ltfs_write_index(vol->label->partid_dp, SYNC_RECOVERY, LTFS_FULL_INDEX, vol); } if (!ret) { ltfsmsg(LTFS_INFO, 17259I, "IP", vol->index->selfptr.partition, (unsigned long long)vol->index->selfptr.block); - ltfs_write_index(vol->label->partid_ip, SYNC_RECOVERY, vol); + ltfs_write_index(vol->label->partid_ip, SYNC_RECOVERY, LTFS_FULL_INDEX, vol); } } else { ltfsmsg(LTFS_ERR, 11231E); @@ -1410,12 +1410,12 @@ int ltfs_write_index_conditional(char partition, struct ltfs_volume *vol) if (partition == ltfs_ip_id(vol) && ! vol->ip_index_file_end) { ltfs_set_commit_message_reason(SYNC_CASCHE_PRESSURE, vol); - ret = ltfs_write_index(partition, SYNC_CASCHE_PRESSURE, vol); + ret = ltfs_write_index(partition, SYNC_CASCHE_PRESSURE, LTFS_FULL_INDEX, vol); } else if (partition == ltfs_dp_id(vol) && (! vol->dp_index_file_end || (vol->ip_index_file_end && vol->index->selfptr.partition == ltfs_ip_id(vol)))) { ltfs_set_commit_message_reason(SYNC_CASCHE_PRESSURE, vol); - ret = ltfs_write_index(partition, SYNC_CASCHE_PRESSURE, vol); + ret = ltfs_write_index(partition, SYNC_CASCHE_PRESSURE, LTFS_FULL_INDEX, vol); } return ret; diff --git a/src/libltfs/periodic_sync.c b/src/libltfs/periodic_sync.c index f11740c7..4edca3ef 100644 --- a/src/libltfs/periodic_sync.c +++ b/src/libltfs/periodic_sync.c @@ -102,7 +102,8 @@ ltfs_thread_return periodic_sync_thread(void* data) } ltfs_set_commit_message_reason(SYNC_PERIODIC, priv->vol); - ret = ltfs_sync_index(SYNC_PERIODIC, true, priv->vol); + + ret = ltfs_sync_index(SYNC_PERIODIC, true, LTFS_INDEX_AUTO, priv->vol); if (ret < 0) { ltfsmsg(LTFS_INFO, 11030I, ret); priv->keepalive = false; diff --git a/src/libltfs/xattr.c b/src/libltfs/xattr.c index b223edd7..e19d9ff3 100644 --- a/src/libltfs/xattr.c +++ b/src/libltfs/xattr.c @@ -882,7 +882,7 @@ static int _xattr_get_virtual(struct dentry *d, char *buf, size_t buf_size, cons } } else if (! strcmp(name, "ltfs.sync")) { ltfs_set_commit_message_reason(SYNC_EA, vol); - ret = ltfs_sync_index(SYNC_EA, false, vol); + ret = ltfs_sync_index(SYNC_EA, false, LTFS_FULL_INDEX, vol); } } @@ -915,8 +915,12 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va size_t size, struct ltfs_volume *vol) { int ret = 0; + enum ltfs_index_type idx_type = LTFS_INDEX_AUTO; - if ((! strcmp(name, "ltfs.commitMessage") || ! strcmp(name, "ltfs.sync")) + if ((! strcmp(name, "ltfs.commitMessage") || + ! strcmp(name, "ltfs.sync") || + ! strcmp(name, "ltfs.vendor.IBM.FullSync") || + ! strcmp(name, "ltfs.vendor.IBM.IncrementalSync")) && d == vol->index->root) { char *value_null_terminated, *new_value; @@ -925,6 +929,17 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va ret = -LTFS_LARGE_XATTR; } +#ifdef FORMAT_SPEC25 + if (! strcmp(name, "ltfs.vendor.IBM.FullSync")) + idx_type = LTFS_FULL_INDEX; + else if (! strcmp(name, "ltfs.vendor.IBM.IncrementalSync")) + idx_type = LTFS_INCREMENTAL_INDEX; +#else + if (! strcmp(name, "ltfs.vendor.IBM.FullSync") || + ! strcmp(name, "ltfs.vendor.IBM.IncrementalSync")) { + } +#endif + ltfs_mutex_lock(&vol->index->dirty_lock); if (! vol->index->dirty) { /* Do nothing because index is clean */ @@ -953,12 +968,12 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va ltfs_set_commit_message_reason_unlocked(SYNC_EA, vol); ltfs_mutex_unlock(&vol->index->dirty_lock); - ret = ltfs_sync_index(SYNC_EA, false, vol); + ret = ltfs_sync_index(SYNC_EA, false, LTFS_FULL_INDEX, vol); return ret; } ret = 0; - /* Update THE commit message in the index */ + /* Update the commit message in the index */ if (vol->index->commit_message) free(vol->index->commit_message); vol->index->commit_message = new_value; @@ -968,7 +983,7 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va } ltfs_mutex_unlock(&vol->index->dirty_lock); - ret = ltfs_sync_index(SYNC_EA, false, vol); + ret = ltfs_sync_index(SYNC_EA, false, idx_type, vol); } else if (! strcmp(name, "ltfs.volumeName") && d == vol->index->root) { char *value_null_terminated, *new_value; @@ -1187,7 +1202,7 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va lock = strtoull(v, &invalid_start, 0); if( (*invalid_start == '\0') && v ) { - mam_lockval new = UNLOCKED_MAM; + mam_lockval_t new = UNLOCKED_MAM; char status_mam[TC_MAM_LOCKED_MAM_SIZE]; switch (vol->t_attr->vollock) { @@ -1241,13 +1256,13 @@ static int _xattr_set_virtual(struct dentry *d, const char *name, const char *va ltfs_set_index_dirty(false, false, vol->index); ltfs_set_commit_message_reason_unlocked(SYNC_ADV_LOCK, vol); - ret = ltfs_sync_index(SYNC_ADV_LOCK, false, vol); + ret = ltfs_sync_index(SYNC_ADV_LOCK, false, LTFS_FULL_INDEX, vol); ret = tape_device_lock(vol->device); if (ret < 0) { ltfsmsg(LTFS_ERR, 12010E, __FUNCTION__); return ret; } - ret = ltfs_write_index(ltfs_ip_id(vol), SYNC_EA, vol); + ret = ltfs_write_index(ltfs_ip_id(vol), SYNC_EA, LTFS_FULL_INDEX, vol); tape_device_unlock(vol->device); } else ret = -LTFS_STRING_CONVERSION; @@ -1501,7 +1516,7 @@ int xattr_set(struct dentry *d, const char *name, const char *value, size_t size if (write_idx) { ltfs_set_commit_message_reason(SYNC_EA, vol); - ret = ltfs_sync_index(SYNC_EA, false, vol); + ret = ltfs_sync_index(SYNC_EA, false, LTFS_INDEX_AUTO, vol); } else ret = 0; diff --git a/src/libltfs/xml_libltfs.h b/src/libltfs/xml_libltfs.h index 2f64e98f..b2e85f70 100644 --- a/src/libltfs/xml_libltfs.h +++ b/src/libltfs/xml_libltfs.h @@ -83,7 +83,7 @@ xmlBufferPtr xml_make_label(const char *creator, tape_partition_t partition, xmlBufferPtr xml_make_schema(const char *creator, const struct ltfs_index *idx); int xml_schema_to_file(const char *filename, const char *creator, const char *reason, const struct ltfs_index *idx); -int xml_schema_to_tape(char *reason, struct ltfs_volume *vol); +int xml_schema_to_tape(char *reason, int type, struct ltfs_volume *vol); /* Functions for reading XML files. See xml_reader_libltfs.c */ int xml_label_from_file(const char *filename, struct ltfs_label *label); diff --git a/src/libltfs/xml_writer_libltfs.c b/src/libltfs/xml_writer_libltfs.c index 5f8f76b1..098b1892 100644 --- a/src/libltfs/xml_writer_libltfs.c +++ b/src/libltfs/xml_writer_libltfs.c @@ -60,6 +60,7 @@ #include "fs.h" #include "tape.h" #include "pathname.h" +#include "inc_journal.h" #include "arch/time_internal.h" /* Structure to control EE's file offset cache and sync file list */ @@ -338,10 +339,11 @@ static int _xml_write_file(xmlTextWriterPtr writer, struct dentry *file, struct /* Write dirty file list */ if (sync_list->fp && file->dirty) { fprintf(sync_list->fp, "%s,%"PRIu64"\n", file->name.name, file->size); - file->dirty = false; sync_list->count++; } + file->dirty = false; + return 0; } @@ -586,6 +588,433 @@ static int _xml_write_schema(xmlTextWriterPtr writer, const char *creator, return 0; } +#ifdef FORMAT_SPEC25 +static int _xml_write_incremental_dir(xmlTextWriterPtr writer, struct dentry *dir) +{ + /* Handle R/O and timestamp if it is dirty */ + if (dir->dirty) { + xml_mktag(xmlTextWriterWriteElement( + writer, BAD_CAST "readonly", BAD_CAST (dir->readonly ? "true" : "false")), -1); + xml_mktag(_xml_write_dentry_times(writer, dir), -1); + } + + /* Handle UID in any case */ + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST UID_TAGNAME, "%"PRIu64, dir->uid), -1); + + /* Handle extended attribute if it is dirty */ + if (dir->dirty) { + xml_mktag(_xml_write_xattr(writer, dir), -1); + dir->dirty = false; + } + + return 0; +} + +static int _xml_open_incremental_dir_ent(xmlTextWriterPtr writer, struct dentry *dir) +{ + /* Handle R/O and timestamp if it is dirty */ + if (dir->dirty) { + xml_mktag(xmlTextWriterWriteElement( + writer, BAD_CAST "readonly", BAD_CAST (dir->readonly ? "true" : "false")), -1); + xml_mktag(_xml_write_dentry_times(writer, dir), -1); + } + + /* Handle UID in any case */ + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST UID_TAGNAME, "%"PRIu64, dir->uid), -1); + + /* Handle extended attribute if it is dirty */ + if (dir->dirty) { + xml_mktag(_xml_write_xattr(writer, dir), -1); + dir->dirty = false; + } + + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "contents"), -1); + + return 0; +} + +static int _xml_open_incrental_root(xmlTextWriterPtr writer, struct ltfs_volume *vol) +{ + int ret = 0; + struct ltfs_index *idx = vol->index; + struct dentry *dir = idx->root; + + /* Handle name tag */ + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "directory"), -1); + if (idx->volume_name.name) { + xml_mktag(_xml_write_nametype(writer, "name", (struct ltfs_name*)(&idx->volume_name)), -1); + } else { + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "name"), -1); + xml_mktag(xmlTextWriterEndElement(writer), -1); + } + + ret = _xml_open_incremental_dir_ent(writer, dir); + + return ret; +} + +static inline int _xml_close_incrental_root(xmlTextWriterPtr writer) +{ + xml_mktag(xmlTextWriterEndElement(writer), -1); /* close contents tag */ + xml_mktag(xmlTextWriterEndElement(writer), -1); /* close directory tag */ + return 0; +} + +static int _xml_open_incremental_dir(xmlTextWriterPtr writer, struct dentry *dir) +{ + int ret = 0; + + /* Handle name tag */ + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "directory"), -1); + xml_mktag(_xml_write_nametype(writer, "name", &dir->name), -1); + xml_mktag(xmlTextWriterEndElement(writer), -1); + + ret = _xml_open_incremental_dir_ent(writer, dir); + + return ret; +} + +static inline int _xml_close_incremental_dir(xmlTextWriterPtr writer) +{ + xml_mktag(xmlTextWriterEndElement(writer), -1); + return 0; +} + +static int _xml_open_incremental_dirs(xmlTextWriterPtr writer, struct incj_path_helper *pm, int offset) +{ + int ret = 0, i = 0; + struct incj_path_element *cur = NULL; + + cur = pm->head; + for (i = 0; i < offset; i++) { + cur = cur->next; + } + + while (cur) { + ret = _xml_open_incremental_dir(writer, cur->d); + if (ret < 0) + break; + cur = cur->next; + } + + return ret; +} + +static int _xml_close_incremental_dirs(xmlTextWriterPtr writer, int pops) +{ + int ret = 0, i = 0; + + for (i = 0; i < pops; i++) { + ret = _xml_close_incremental_dir(writer); + if (ret < 0) + break; + } + + return ret; +} + +static int _xml_write_incremental_delete(xmlTextWriterPtr writer, enum journal_reason reason, struct ltfs_name *name) +{ + /* Open directory tag or file tag based on the provided reason */ + switch (reason) { + case DELETE_DIRECTORY: + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "directory"), -1); + break; + case DELETE_FILE: + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "file"), -1); + break; + default: + ltfsmsg(LTFS_ERR, 17304E, reason); + return -1; + break; + } + + /* Create a name tag */ + xml_mktag(_xml_write_nametype(writer, "name", name), -1); + + /* Create a empty deleted tag */ + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "deleted"), -1); + xml_mktag(xmlTextWriterEndElement(writer), -1); + + /* Close directory or file tag */ + xml_mktag(xmlTextWriterEndElement(writer), -1); + + return 0; +} + +static int _xml_goto_increment_parent(xmlTextWriterPtr writer, + struct jentry *ent, struct incj_path_helper **cur, + struct ltfs_volume *vol) +{ + int ret = 0, matches = 0, pops = 0; + bool perfect_match = false; + char *parent_path = NULL, *filename = NULL; + struct incj_path_helper *new = NULL; + + parent_path = strdup(ent->id.full_path); + if (!parent_path) { + ltfsmsg(LTFS_ERR, 10001E, "parent path for traveling incremental index dirs"); + return -LTFS_NO_MEMORY; + } + + fs_split_path(parent_path, &filename, strlen(parent_path) + 1); + if (strlen(parent_path) == 0) { + parent_path = "/"; + } + + ret = incj_create_path_helper(parent_path, &new, vol); + if (ret < 0) { + free(parent_path); + return ret; + } + + ret = incj_compare_path(*cur, new, &matches, &pops, &perfect_match); + if (ret < 0) { + incj_destroy_path_helper(new); + free(parent_path); + return ret; + } + + if (pops) { + ret = _xml_close_incremental_dirs(writer, pops); + if (ret < 0) { + incj_destroy_path_helper(new); + free(parent_path); + return ret; + } + } + + if (!perfect_match) + ret = _xml_open_incremental_dirs(writer, new, matches); + + if (!ret) { + incj_destroy_path_helper(*cur); + *cur = new; + } else { + incj_destroy_path_helper(new); + } + + if (strcmp(parent_path, "/")) + free(parent_path); + + return ret; +} + +/** + * Write directory tags for incremental index based on current incremental journal information + * @param writer output pointer + * @param vol ltfs volume + * @param offset_c file pointer to write offest cache + * @param sync_list file pointer to write sync file list + * @return 0 on success or negative on failure + */ +static int _xml_write_inc_journal(xmlTextWriterPtr writer, struct ltfs_volume *vol, + struct ltfsee_cache* offset_c, struct ltfsee_cache* sync_list) +{ + int ret = 0; + bool failed = false; + struct ltfsee_cache offset = {NULL, 0}; /* Cache structure for file offset cache */ + struct ltfsee_cache list = {NULL, 0}; /* Cache structure for sync list */ + struct jentry *ent = NULL, *tmp = NULL; + struct incj_path_helper *cur_parent = NULL; + + /* Create a directory tag for root */ + ret = _xml_open_incrental_root(writer, vol); + if (ret < 0) { + return ret; + } + + ret = incj_create_path_helper("/", &cur_parent, vol); + if (ret < 0) { + return ret; + } + + /* Crawl incremental journal and generate XML tags */ + incj_sort(vol); + HASH_ITER(hh, vol->journal, ent, tmp) { + if (!failed) { + ret = _xml_goto_increment_parent(writer, ent, &cur_parent, vol); + if (!ret) { + switch (ent->reason) { + case CREATE: + /* TODO: Need to support sync cache and offset cache */ + if (ent->dentry->isdir) { + /* Create XML recursively */ + ret = _xml_write_dirtree(writer, ent->dentry, vol->index, &offset, &list); + } else { + /* Create XML for a file */ + ret = _xml_write_file(writer, ent->dentry, &offset, &list); + } + break; + case MODIFY: + if (ent->dentry->isdir) { + /* Create XML for dir (no recursive) */ + ret = _xml_write_incremental_dir(writer, ent->dentry); + } else { + /* Create XML for a file */ + ret = _xml_write_file(writer, ent->dentry, &offset, &list); + } + break; + case DELETE_FILE: + case DELETE_DIRECTORY: + /* Create a delete tag */ + ret = _xml_write_incremental_delete(writer, ent->reason, &ent->name); + break; + default: + ltfsmsg(LTFS_ERR, 17303E, ent->reason); + ret = -LTFS_UNEXPECTED_VALUE; + break; + } + + if (ret < 0) + failed = true; + } else + failed = true; + } + HASH_DEL(vol->journal, ent); + incj_dispose_jentry(ent); + } + + if (cur_parent) { + incj_destroy_path_helper(cur_parent); + } + + /* Clear created directory just in case */ + incj_clear(vol); + + /* Close the directory tag for root */ + ret = _xml_close_incrental_root(writer); + + return ret; +} + +/** + * Generate an XML schema, sending it to a user-provided output (memory or file). + * Note: this function does very little input validation; any user-provided information + * must be verified by the caller. + * @param writer the XML writer to send output to + * @param priv LTFS data + * @param pos position on tape where the schema will be written + * @return 0 on success, negative on failure + */ +static int _xml_write_incremental_schema(xmlTextWriterPtr writer, const char *creator, struct ltfs_volume *vol) +{ + int ret; + size_t i; + char *update_time; + struct ltfsee_cache offset = {NULL, 0}; /* Cache structure for file offset cache */ + struct ltfsee_cache list = {NULL, 0}; /* Cache structure for sync list */ + struct ltfs_index *idx = vol->index; + + ret = xml_format_time(idx->mod_time, &update_time); + if (!update_time) + return -1; + else if (ret == LTFS_TIME_OUT_OF_RANGE) + ltfsmsg(LTFS_WARN, 17224W, "modifytime", (unsigned long long)idx->mod_time.tv_sec); + + ret = xmlTextWriterStartDocument(writer, NULL, "UTF-8", NULL); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17057E, ret); + return -1; + } + + xmlTextWriterSetIndent(writer, 1); + /* Define INDENT_INDEXES to write Indexes to tape with full indentation. + * This is normally a waste of space, but it may be useful for debugging. */ +#ifdef INDENT_INDEXES + xmlTextWriterSetIndentString(writer, BAD_CAST " "); +#else + xmlTextWriterSetIndentString(writer, BAD_CAST ""); +#endif + + /* write index properties */ + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "ltfsincrementalindex"), -1); + xml_mktag(xmlTextWriterWriteAttribute(writer, BAD_CAST "version", + BAD_CAST LTFS_INDEX_VERSION_STR), -1); + xml_mktag(xmlTextWriterWriteElement(writer, BAD_CAST "creator", BAD_CAST creator), -1); + if (idx->commit_message && strlen(idx->commit_message)) { + xml_mktag(xmlTextWriterWriteFormatElement(writer, BAD_CAST "comment", + "%s", BAD_CAST (idx->commit_message)), -1); + } + xml_mktag(xmlTextWriterWriteElement(writer, BAD_CAST "volumeuuid", BAD_CAST idx->vol_uuid), -1); + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST "generationnumber", "%u", idx->generation), -1); + xml_mktag(xmlTextWriterWriteElement(writer, BAD_CAST "updatetime", BAD_CAST update_time), -1); + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "location"), -1); + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST "partition", "%c", idx->selfptr_inc.partition), -1); + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST "startblock", "%"PRIu64, idx->selfptr_inc.block), -1); + xml_mktag(xmlTextWriterEndElement(writer), -1); + if (idx->backptr.block) { + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "previousgenerationlocation"), -1); + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST "partition", "%c", idx->backptr.partition), -1); + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST "startblock", "%"PRIu64, idx->backptr.block), -1); + xml_mktag(xmlTextWriterEndElement(writer), -1); + } + + if (idx->backptr_inc.block) { + xml_mktag(xmlTextWriterStartElement(writer, BAD_CAST "previousincrementallocation"), -1); + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST "partition", "%c", idx->backptr_inc.partition), -1); + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST "startblock", "%"PRIu64, idx->backptr_inc.block), -1); + xml_mktag(xmlTextWriterEndElement(writer), -1); + } + + xml_mktag(xmlTextWriterWriteFormatElement( + writer, BAD_CAST NEXTUID_TAGNAME, "%"PRIu64, idx->uid_number), -1); + + { + char *value = NULL; + + switch (idx->vollock) { + case LOCKED_MAM: + asprintf(&value, "locked"); + break; + case PERMLOCKED_MAM: + asprintf(&value, "permlocked"); + break; + default: + asprintf(&value, "unlocked"); + break; + } + + if (value) + xml_mktag(xmlTextWriterWriteElement(writer, BAD_CAST "volumelockstate", BAD_CAST value), -1); + + free(value); + } + + /* Create XML of update */ + xml_mktag(_xml_write_inc_journal(writer, vol, &offset, &list), -1); + + /* Save unrecognized tags */ + if (idx->tag_count > 0) { + for (i=0; itag_count; ++i) { + if (xmlTextWriterWriteRaw(writer, idx->preserved_tags[i]) < 0) { + ltfsmsg(LTFS_ERR, 17092E, __FUNCTION__); + return -1; + } + } + } + + xml_mktag(xmlTextWriterEndElement(writer), -1); + ret = xmlTextWriterEndDocument(writer); + if (ret < 0) { + ltfsmsg(LTFS_ERR, 17058E, ret); + return -1; + } + + free(update_time); + return 0; +} +#endif + /************************************************************************************** * Global Functions **************************************************************************************/ @@ -809,10 +1238,12 @@ int xml_schema_to_file(const char *filename, const char *creator, /** * Generate an XML Index based on the vol->index->root directory tree. * The generated data are written directly to the tape with the appropriate blocksize. + * @param reason the reason of writing an index on tape + * @param type index type to write (shall be LTFS_FULL_INDEX or LTFS_INCREMENTAL_INDEX) * @param vol LTFS volume. * @return 0 on success or a negative value on error. */ -int xml_schema_to_tape(char *reason, struct ltfs_volume *vol) +int xml_schema_to_tape(char *reason, int type, struct ltfs_volume *vol) { int ret, bk = -1; xmlOutputBufferPtr write_buf; @@ -875,7 +1306,19 @@ int xml_schema_to_tape(char *reason, struct ltfs_volume *vol) /* Generate the Index. */ asprintf(&creator, "%s - %s", vol->creator, reason); if (creator) { - ret = _xml_write_schema(writer, creator, vol->index); + switch (type) { + case LTFS_FULL_INDEX: + ret = _xml_write_schema(writer, creator, vol->index); + break; +#ifdef FORMAT_SPEC25 + case LTFS_INCREMENTAL_INDEX: + ret = _xml_write_incremental_schema(writer, creator, vol); + break; +#endif + default: + ret = -LTFS_BAD_INDEX_TYPE; + break; + } if (ret < 0) { ltfsmsg(LTFS_ERR, 17055E, ret); } diff --git a/src/ltfs_fuse.c b/src/ltfs_fuse.c index 00aaf3bc..4e12188f 100644 --- a/src/ltfs_fuse.c +++ b/src/ltfs_fuse.c @@ -452,7 +452,7 @@ int ltfs_fuse_release(const char *path, struct fuse_file_info *fi) ret = ltfs_fsops_close(file->file_info->dentry_handle, dirty, open_write, true, priv->data); if (write_index) { ltfs_set_commit_message_reason(SYNC_CLOSE, priv->data); - ltfs_sync_index(SYNC_CLOSE, true, priv->data); + ltfs_sync_index(SYNC_CLOSE, true, LTFS_INDEX_AUTO, priv->data); } _file_close(file->file_info, priv); diff --git a/src/utils/ltfsck.c b/src/utils/ltfsck.c index 007ae6ce..dc7cef85 100644 --- a/src/utils/ltfsck.c +++ b/src/utils/ltfsck.c @@ -1154,7 +1154,7 @@ int _rollback_ip(struct ltfs_volume *vol, struct other_check_opts *opt, struct t if (ret != LTFSCK_NO_ERRORS) ltfsmsg(LTFS_ERR, 16059E, ret); } else { - ret = ltfs_write_index(ltfs_ip_id(vol), SYNC_ROLLBACK, vol); + ret = ltfs_write_index(ltfs_ip_id(vol), SYNC_ROLLBACK, LTFS_FULL_INDEX, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 16060E, ret); ret = LTFSCK_OPERATIONAL_ERROR; @@ -1176,7 +1176,7 @@ int _rollback_dp(struct ltfs_volume *vol, struct other_check_opts *opt, struct t ltfsmsg(LTFS_ERR, 16055E, ret); } else { ltfs_set_commit_message_reason(SYNC_ROLLBACK, vol); - ret = ltfs_write_index(ltfs_dp_id(vol), SYNC_ROLLBACK, vol); + ret = ltfs_write_index(ltfs_dp_id(vol), SYNC_ROLLBACK, LTFS_FULL_INDEX, vol); if (ret < 0) { ltfsmsg(LTFS_ERR, 16056E, ret); ret = LTFSCK_OPERATIONAL_ERROR; From 29b6f3457639c9544d7e72891268c9d69b73480f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Julio=20C=C3=A9sar=20Cel=C3=B3n=20Altamirano?= <37186598+juliocelon@users.noreply.github.com> Date: Wed, 31 Jul 2024 15:36:30 -0600 Subject: [PATCH 120/121] Remove CentOS7 and RHEL7 Build for Travis in master branch (#477) --- .github/workflows/build-centos7.yml | 17 ----------------- README.md | 4 ---- 2 files changed, 21 deletions(-) delete mode 100644 .github/workflows/build-centos7.yml diff --git a/.github/workflows/build-centos7.yml b/.github/workflows/build-centos7.yml deleted file mode 100644 index 2cdcce57..00000000 --- a/.github/workflows/build-centos7.yml +++ /dev/null @@ -1,17 +0,0 @@ -name: CentOS7 Build Job - -on: [push, pull_request] - -jobs: - build: - name: Build on CentOS7 - runs-on: ubuntu-latest - - steps: - - name: Set up Git repository - uses: actions/checkout@v1 - - name: Build LTFS - id: build - uses: LinearTapeFileSystem/CentOS7-Build@v1.4 - with: - destination: '/tmp/ltfs' diff --git a/README.md b/README.md index 4b504d57..59f03049 100644 --- a/README.md +++ b/README.md @@ -141,12 +141,8 @@ In some systems, you might need `sudo ldconfig -v` after `make install` to load |:----------------------------------:|:-------:|:--------------------------------------------------------------------------------------------------------------------------------:| | RHEL 8 | x86\_64 | OK - Not checked automatically | | RHEL 8 | ppc64le | OK - Not checked automatically | - | RHEL 7 | x86\_64 | OK - Not checked automatically | - | RHEL 7 | ppc64le | OK - Not checked automatically | | CentOS 8 (Rocky Linux) | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS8%20Build%20Job/badge.svg?branch=master) | | CentOS 8 (Rocky Linux) | ppc64le | OK - Not checked automatically | - | CentOS 7 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/CentOS7%20Build%20Job/badge.svg?branch=master) | - | CentOS 7 | ppc64le | OK - Not checked automatically | | Fedora 28 | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Fedora28%20Build%20Job/badge.svg?branch=master) | | Ubuntu 16.04 LTS | x86\_64 | ![GH Action status](https://github.com/LinearTapeFileSystem/ltfs/workflows/Ubuntu%2016.04%20Build%20Job/badge.svg?branch=master) | | Ubuntu 16.04 LTS | ppc64le | OK - Not checked automatically | From 627484b2b0bbcdaac30281560a1a52601823336b Mon Sep 17 00:00:00 2001 From: Missael Palacios Date: Mon, 26 Aug 2024 20:20:16 -0600 Subject: [PATCH 121/121] Changed flags for _ltfs_search_index_wp --- src/libltfs/ltfs.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libltfs/ltfs.c b/src/libltfs/ltfs.c index baa7aae3..7b74b37b 100644 --- a/src/libltfs/ltfs.c +++ b/src/libltfs/ltfs.c @@ -1673,7 +1673,8 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec (unsigned long long)vol->dp_coh.volume_change_ref, (unsigned long long)volume_change_ref); - ret = _ltfs_search_index_wp(recover_symlink, false, &seekpos, vol); + /* Index of IP could be corrupted. So set skip flag */ + ret = _ltfs_search_index_wp(recover_symlink, true, &seekpos, vol); if (ret < 0) goto out_unlock; @@ -1702,8 +1703,7 @@ int ltfs_mount(bool force_full, bool deep_recovery, bool recover_extra, bool rec (unsigned long long)vol->dp_coh.volume_change_ref, (unsigned long long)volume_change_ref); - /* Index of IP could be corrupted. So set skip flag */ - ret = _ltfs_search_index_wp(recover_symlink, true, &seekpos, vol); + ret = _ltfs_search_index_wp(recover_symlink, false, &seekpos, vol); if (ret < 0) goto out_unlock;