PLScsi/  Tools/  pldd/

Updated on Friday, July 13, 2004

Features coming soon (write to encourage us, if you care):

Mac OS X

Pseudo-random bit source (more thruput than Linux/ Mac and entirely new in Windows).

News: Linux pldd since July 2004 attempts ioctl SG_IO on any /dev/ name, to make device names like /dev/hdd and /dev/scd0 work, which first became possible in Linux 2.6.

Free Download:

pldd.tar.gz for Linux

pldd.zip in case you dislike .tar.bz2

pldd.exe for Win 2K/ XP

 

See Also:

open issues

links to this page

knoppix faq

pldd of usr proc bin at google

linux sg utils home page

 

Free Preview:

lddio.c.txt

wddio.cpp.txt 

 

Free Help re Unix:

man pages dd at google

man pages cmp at google

man pages null at google

man pages urandom at google

man pages zero at google

 

Free Help re Scsi Pass Thru:

tools/ gccscsi

wscsi.dll for Win 2K/ XP

 

Free Help re Linux C:

man pages fopen64 at google

man pages fseeko64 at google

kernelnewbies archive at google

kernelnewbies archive at marc

 

Free gcc/g++ for Windows:

mingw win gcc/g++ faq

 

Free Help re Microsoft C:

msdn.microsoft C __int64 at google

msdn.microsoft %I64 at google

msdn.microsoft GetStdHandle at google

msdn.microsoft CreateFile at google

msdn.microsoft LARGE_INTEGER at google

msdn.microsoft SetFilePointer at google

msdn.microsoft ReadFile at google

msdn.microsoft WriteFile at google

msdn.microsoft GetTickCount at google

msdn.microsoft SetConsoleCtrlHandler at google


 

Please do back up your hard drive before you run pldd, rather than having pldd demonstrate your need to backup more often.


1. Readme

Our pldd is a rewrite of dd, the standard Unix tool for copying the bytes of files and block devices.  With our pldd, you may:

seek/ read/ write TiB via Scsi pass thru, and

work in hex and seek to lba's, and

say late is wrong.

All that works because our pldd understands more than the standard man dd options:

if= skip= of= seek= bs= count= --version --help

pldd works today in Linux 2.6/ 2.4 and Windows XP/ 2K.  pldd and plscsi are layers built on top of gccscsi.  gccscsi now exists also for Mac OS X "authoring devices" and more robustly inMac OF (Open Firmware) (boot Forth), Win 9X/ME, and Dos.  Bizarrely enough, the Mac OS X gccscsi demo appears at the plscsi/mac page, not at the gccscsi page.  Please do write to encourage us if you want to see pldd and plscsi and gccscsi work in more contexts.

pldd has more than standard options.  After the fact we note that in -l we may have reinvented the standard man dd option noerror,but then also in pldd we see:

the prefixes 0x and x mean hex, rather than the default decimal

the suffixes k m g t mean Ki Mi Gi Ti i.e. kebi mebi gibi tebi

sbs= means bytes per block for skip= and seek=, overriding the default sbs=bs

ms= means limit milliseconds per copy of bs=, instead of the default indefinite (or 28 hour) wait

-l, prototyped as -f, means list troubles, which means copy past error or eof, which makes sense only in combination with count=, first implemented as -f

-v means verbose, which for Scsi pass thru means print the CDB's

We regret, as yet:

we have written pldd only for Linux and for Win 2K/SP and some for Mac OS X, and

we copy with only one thread, not two, and

we support only the options of standard dd that we use, and

Linux/ Mac OS X/ Solaris disagree over which dd suffixes mean Ki Mi Gi Ti, and

SCSI pass thru doesn't work unless the hdd/ dvd/ cd op x25 READ_CAPACITY works.

In practice we have never yet discovered a SCSI pass thru that does support op x28/2A READ_10/ WRITE_10 without also supporting op x25 READ_CAPACITY to determine bytes per block.  If you do discover such a thing, please write Pat LaVarre to tell us of it.  We have heard of DVD/ CD commonly getting the maxLba off by one in the READ_CAPACITY data, but we have never yet heard of anyone getting the bytes per block wrong.


2. FAQ

2.1 Why

2.1.1. Why is pldd still running, seven seconds after I specified ms=7000?

When you specify ms=, pldd only times each copy of bs=.  pldd does not timeout and cancel each copy.  After a slow copy of bs= completes, pldd pretends it saw an i/o error.  pldd then halts the whole copy only if you did not specify -f.

2.1.2 Why doesn't pldd say K = 1000 and Ki = 1024 like the US NIST does?

We see the standard dd of Linux/ Mac OS X/ Solaris mutually disagree over which suffixes mean Ki Mi Gi Ti and which mean K M G T.  Since there is no standard, we have added our easy-to-type k m g t standard.

pldd thinks the suffixes k m g t mean Ki Mi Gi Ti i.e. x400 x10:0000 4000:0000 100:0000:0000 ...  = 1,024 1,048,576 1,073,741,824 1,099,511,627,776 i.e. round numbers in hex that approximate powers of decimal 1000 with geometric inaccuracy.

2.1.3. Why did the Scsi pass thru fail?

pldd distinguishes Four ways a Scsi pass thru may fail:

  1. The system may refuse the pass thru, e.g. if you lack root privilege, or if a timeout occurs.

  2. The system may claim to have copied more data bytes than allotted, or less than zero bytes.

  3. The system may claim to have copied more auto sense bytes than allotted, or less than zero bytes.

  4. An auto sensed failure may have occurred, with or without copying all data bytes allotted.

2.2 How

Q1. How do I see the CDB's authored by pldd?

Say -v when if= and/or of= is a Scsi pass thru device.

Q2. How do I skip=/ seek= to an lba?

Say sbs= bytes per disk block.  Likely sbs=2k for dvd/cd and sbs=512 for hdd.  Saying sbs=512 will let you skip=/ seek= past a count of 0.5 KiB blocks even on a dvd/cd disc, provided you skip=/ seek= to a multiple of four (since 4 = 2 Ki / 512).

Q3. How do I read/ write past the first error?

Say -l (except for now you have to say -f instead) and say count= how many times you do wish to copy bs=.

Q5. How do I say late is wrong?

Say ms=.


3. Source

Download the source by choosing whichever you prefer of the links to .tar.bz2 and .zip above left.  You will get a trivial Makefile for Linux, and equally trivial mak.bat for Windows, and ten source files.

We found we had to write our source code in Three incompatible dialects of C:

In the C of the gcc/g++ of Linux, we wrote:

pldd.h = help all the .c* link together

version.h = date the other source files

pldd.c = run from the command line

lddio.c = copy TiB via file or block device, or Scsi pass thru

lscsi.c = pass thru Scsi

In the C++ of MinGW gcc/g++ for Windows, we wrote:

ntddscsi.hpp = help link Windows C of pass thru Scsi

wddio.cpp = copy TiB via file or block device, or Scsi pass thru

wscsi.cpp = pass thru Scsi

In the C++ of Microsoft Visual Studio for Windows, we wrote:

pldd.hpp= help all the .c* link together

pldd.cpp = run from the command line

Executables created by Linux `make` or by Windows `mak.bat` include:

Linux:

pldd = Linux console app

lscsi.so = Linux shared library

Win 2K/XP:

pldd.exe = Win 2K/XP console app

wscsi.ddl = Windows shared library


4. Bugs

We allude cryptically to our known bugs in the commentary that begins each .h or .c file.

In the July 13, 2004 version we may have said:

pldd.h/ pldd.hpp:

Define LIL(I, N) incorrectly if cpu is big-endian.

pldd.c/ pldd.cpp:

Abbreviate if=/dev/zero without abbreviating if=/dev/urandom

Lose count of copies if halted via Ctrl+C.

Copy with one thread, not two.

Die when bs= exceeds INT_MAX.

Do not guess whether K M T G means Ki Mi Ti Gi or not.

Always exit(0).

lddio.c/ wddio.cpp:

Pass only block-aligned access thru Scsi.

Limit bytes per block and per i/o to UINT_MAX = max size_t.

lddio.c:

Try SG_IO if and only if /dev/ begins the device name.

Speak only thru Gnu/ Linux signed 64-bit i/o and SG_IO.

Hang when asked to exceed size of block device.

wddio.cpp:

Try SG_IO only if \\.\ begins the device name.

Speak only thru Windows signed 64-bit i/o and IOCTL_SCSI_PASS_THROUGH.

lscsi.c/ wscsi.cpp:

Never ask to time out in less than 28 hours,

Never ask for other than x12 sense bytes.

Never return sense bytes, always print in hex to stderr.

Refuse to pass thru more than xFF bytes of CDB.

Limit bytes per CDB to what caller can allocate virtually.

Don't bother to filter duplicate lines from dumps.

wscsi.cpp:

Never use IOCTL_SCSI_PASS_THROUGH_DIRECT, not even for > 16 KiB data.

Take the time to invoke calloc and free once per cdb.

64-bit Windows may choke via cast between int and HANDLE thru __int64.


5. Examples

The examples here appear expressed in terms of the Linux device name /dev/sg0.  But other SCSI pass thru device names also work, such as the \\.\A: and \\.\TAPE0 device names of Windows, also the /dev/hdd and /dev/scd0 device names of Linux.  `plscsi -w` tries (but does not yet wholly succeed) in guessing for you what SCSI pass thru device names exist for you.

5.1 Read 1 MiB from a specific hex LBA.

$ pldd -v if=/dev/sg0 bs=1m sbs=2k skip=0x123 of=/dev/null count=1
cdb: x 25 00 00 00 00 00 00 00 00 00 .. .. .. .. .. .. "%@@@@@@@@@"
in:: x 00 CE 29 FF 00 00 08 00 .. .. .. .. .. .. .. .. "@N)?@@H@"
cdb: x 28 00 00 00 01 23 00 02 00 00 .. .. .. .. .. .. "(@@@A#@B@@"
fwrite x100000 bytes
$

5.2 Write past end of disk.

$ pldd of=/dev/sg0 bs=1m if=/dev/zero
...
x 2A 00 00 28 00 00 00 02 00 00 .. .. .. .. .. .. "*@@N*@@@ @"
x F0 00 05 00 28 00 00 18 00 00 00 00 21 00 00 00 "p@E@N*@X@@@@!@@@"
x 00 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .. "@@"
5368709120 = x140000000 bytes copied
$

5.3 Read past slow spots and past errors.

$ pldd -f if=/dev/sg0 bs=1m of ms=1000
...
slow 1.090s
3145728 = 0x300000 bytes copied
...
x 28 00 00 12 34 00 00 02 00 00 .. .. .. .. .. .."(@@<^@@B@@"
x F0 00 03 00 12 34 00 18 00 00 00 00 11 00 00 00 "p@C@<_+X@@@@QB@@"
x 00 00 .. .. .. .. .. .. .. .. .. .. .. .. .. .."@@"
slow 20.733s
2442133504 = 0x91900000 bytes copied
...

5.4 Create a file containing 8 GiB of zeroes.

$ pldd -v if bs=1m of=dd.bin seek=x1FFF count=1
$

6. Versus Linux sg_utils

For Windows we have found no tool comparable to pldd.

For Linux, we have found no tool with options comparable to the -l -v ms= sbs= of pldd.   But in search of the dd-compatible subset of pldd we did follow the web trail:

http://www.tldp.org/HOWTO/SCSI-Generic-HOWTO/sg3_utils.html
Appendix A. Sg3_utils package
variants of the Unix dd command: sg_dd,
sgp_dd, sgq_dd and sgm_dd, ...
...

http://www.torque.net/sg/

http://www.torque.net/sg/p/sg3_utils-1.05.tgz
$ tar -zx ...
$ cd
sg3_utils-1.05
$ make
$ sudo ./sg_scan -i ...
$ sudo ./sg_dd of=/dev/sg1 if=/dev/zero bs=2k bpt=512 count=1024 ...

The sg utils doc says sgp_dd is sg_dd except sgp_dd copies by reading and writing in parallel rather than reading and writing in sequence.

For how we use *dd, pldd works more like dd
and sg_dd works less like dd in Three significant ways:

1) In dd and in pldd, bytes per copy is bs=.  In sg_dd, bytes per copy is bs= multiplied by bpt=.

2) In dd and in pldd, an unspecified count= means copy til error.  In sg_dd, an unspecified count= means copy up to the maxLba reported by op x25 READ_CAPACITY.

3)In dd and in pldd, both if= and of= may be files or block devices.  The sg utils doc says in sg_dd either if= or of= or both have to be SG_IO devices.

For how we use *dd, the remaining Six loud differences maybe don't much matter:

1) pldd and sg_dd use -D_LARGEFILE64_SOURCE.  Only sg_dd uses -D_FILE_OFFSET_BITS=64.  Whether that is a bug in our pldd for Linux, we do not yet know.

2) pldd sends op x25 READ_CAPACITY to discover bytes per disk block.  In sg_dd, bytes per disk block is bs=.

3) pldd understands the sbs= multiplier for seek= and skip=, distinct from the bs= ibs= obs= of dd and of sg_dd.

4) pldd understands the 0x and x prefixes for hex numbers, dd and sg_dd do not.  The many *dd disagree over how to say Ki Mi Gi Ti, pldd understands the suffixes k m g t, as does sg_dd.

5) pldd and sg_dd agree on the need for a shorthand for of=/dev/null.  The sg_dd shorthand is of=., the pldd shorthand is of without an = sign.

6) Only pldd defines a shorthand (specifically if without an = sign) to mean if=/dev/zero.

We notice none of the *dd define shorthand to mean /dev/urandom or /dev/random.


7. Authors

Pat LaVarre typed the source code, but

thanks Steve Olney for the translation to Windows C++ from Linux C, and

thanks John McKell for -v, if= of= block device, and clear commentary for calculating exitInt.


8. Changes

FUTURE:  Please write Pat LaVarre to tell us if you wish already we also supported:

Win 2K/SP IOCTL_SCSI_PASS_THROUGH_DIRECT above 16 KiB/cdb

Win 9X/ME, or Dos, or Mac OF (boot Forth), or Mac OS X

Friday, July 13, 2004

Revise Linux to try ioctl with device names that began working only in Linux 2.6/ 2.5.  Offer .tar.gz rather than .tar.bz2 for Linux/ Mac.

Update Mac OS X SCSI pass thru status.  Mention features coming soon.

Mention more example SCSI pass thru device names.  Separate the offer of a Windows .dll for download from the pldd.exe that always has run without .dll.

Thursday, October 9, 2003

Mention dd noerror, the Mac OS X rewrite, and Google glitch of thinking gccscsi is here.

Link to msdn.microsoft SetConsoleCtrlHandler at Google

Saturday, September 27, 2003

Link to man pages cmp and to msdn.microsoft GetTickCount.

Polish the html.  Move mingw faq to ../mingw/.

Thursday, September 25, 2003

Update version.h and compressed .html and do not segfault Linux if input length is < bs (common with stdin).

Tuesday, September 23, 2003:

Let ../gccscsi present the Links and Preview for Scsi Pass Thru.  Focus here on the 64-bit i/o that breaks thru the 2 to 4 GiB per file barrier.

Saturday, September 20, 2003

Run in Win 2K/XP, not just Linux.

Thursday, September 18, 2003

Define sbs= to override the default bs= bytes per block of skip= and seek=.

Define ms= to limit milliseconds per copy of bs=.

Define -v to mean trace cdb/ file read/ write.

Define what should be -l but is -f to mean copy past error or eof up to count=,

Wednesday, September 17, 2003

Link to unrelated pldd of usr proc bin.

Choose read/write not SG_IO for the block devices of Linux such as /dev/scd*.

Never read/write zero blocks.

Clean up file permissions in the .tar.bz2 compressed folder.

Clean up whitespace in source.

Friday, September 12, 2003

Link to comparable sg_dd sgp_dd etc. of sg_utils of sg howto.

Fetch bytes per block via op x25 READ_CAPACITY, for when != 2 Ki.

Comment the four ways Scsi pass thru may fail.

Tuesday, September 9, 2003:

Publish!

Posted by Pat LaVarre