Low level hard disk programming 
Author Message
 Low level hard disk programming

I looked for information about hard disk's low level format on various
groups (eg. comp.lang.asm.x86,comp.os.linux), but my search only took
me to posts like: "dont do it", "it's hardware vendor specific", "the
new ide doesnt make a real low level format".
Now, I will make a few questions that involves low level programming
and I like to see professional answers, not the simple answers like
the above mentioned.
Then, my questions are:
- How is implemented the Int13,function 5 actually for hard disks?
- How is implemented the bad sector marking on a hard disk? (not the
FFF7h cluster mark in the FAT), but the real table management that
knows which physical sector is really bad, something like the "track
address fields" for a floppy disk format.
- How does the BIOS low level format work and which are the supported
disks (it means, vendor, capacity, etc).
- ...
I have other doubts, but first of all I want to know if there is
anybody that really can help me.
Thanks!


Tue, 01 Feb 2005 11:21:27 GMT  
 Low level hard disk programming

Quote:

>I looked for information about hard disk's low level format on various
>groups (eg. comp.lang.asm.x86,comp.os.linux), but my search only took
>me to posts like: "dont do it", "it's hardware vendor specific", "the
>new ide doesnt make a real low level format".

But you don't believe them?  What would it take to convince you?

Quote:
>Now, I will make a few questions that involves low level programming
>and I like to see professional answers, not the simple answers like
>the above mentioned.
>Then, my questions are:
>- How is implemented the Int13,function 5 actually for hard disks?

Look, if you don't believe us, go disassemble the damn BIOS yourself.  It
isn't that hard.  You'll find that the controller in an IDE drive does a
lot of the work independently.

You could also go download a Linux kernel and read through the disk driver
code.

Quote:
>- How is implemented the bad sector marking on a hard disk? (not the
>FFF7h cluster mark in the FAT), but the real table management that
>knows which physical sector is really bad, something like the "track
>address fields" for a floppy disk format.

There are spare sectors on each track of an IDE disk.  At the factory, they
scan the disk looking for physical bad blocks.  If bad blocks are found,
there is a table on the track that tells it which sectors are bad, and the
drive substitutes some of the extra sectors instead.  The microcode that
controls all of this is usually stored in a special track on the hard disk,
and is loaded into the drive's controller upon spin-up.

Unless you rewrite the controlware, you can't dink with the physical bad
block table.
--

  Providenza & Boekelheide, Inc.



Tue, 01 Feb 2005 13:56:02 GMT  
 Low level hard disk programming

Quote:
> I looked for information about hard disk's low level format on various
> groups (eg. comp.lang.asm.x86,comp.os.linux), but my search only took
> me to posts like: "dont do it", "it's hardware vendor specific", "the
> new ide doesnt make a real low level format".

> Now, I will make a few questions that involves low level programming
> and I like to see professional answers, not the simple answers like
> the above mentioned.
> Then, my questions are:
> - How is implemented the Int13,function 5 actually for hard disks?

Easy answer: disassemble your BIOS and see.

Slightly longer, and probably dated, answer:
[] Do some stuff to make sure the drive isn't busy (and exists etc), like
checking the status port
[] Load port 1F2h with the number of sectors to format
[] Load port 1F3h with the starting sector number
[] Load port 1F4h with the low cylinder number
[] Load port 1F5h with the high cyclinder number
[] Set sector size and head vie port 1F6
[] Maybe set write precompensation (port 1F1h), maybe not if the controller
doesn't need it
[] Fire command 50h (format track) to port 1F7h
[] Fire the sector data table (identical to the data passed to function 5
int 13h) to the controller via port 1F0h (in words)

This is a summary of what is done, there are various checks needed to make
sure you can go onto the next stage, and that the data you are sending to
your drive is correct. Get hold of a copy of The Undocumented PC for a much
more complete description, or there's probably many sites on the net that
will help. Or read the Linux kernel source.

I don't see whay people would say "don't do it" for this question ...

Quote:
> - How is implemented the bad sector marking on a hard disk? (not the
> FFF7h cluster mark in the FAT), but the real table management that
> knows which physical sector is really bad, something like the "track
> address fields" for a floppy disk format.

There's two layers of this. One is done automatically by the HDD itself,
before any data is actually put onto the cable. You control this by what you
send to the controller in the sector data table. How this is implemented
depends a lot on the disk (sorry). Maybe it's stored in flash RAM on the
HDD, maybe it's put onto the disk in a special spot, maybe it's done
completely differently. All that you can know is that you tell it to remap
the sector, and somehow it magically does it. What happens from there is not
known (nor important, unless you are into forensics or something).

The second is putting a "bad sector" mark on the sector. This does not fix
it, but just writes a byte to the sector data that when read back tells the
HDD to report it as a bad sector. This effectively does nothing apart from
provide one bit of storage per sector, for you to use in whatever way you
want (however it should be used for bad sector marking).

Quote:
> - How does the BIOS low level format work and which are the supported
> disks (it means, vendor, capacity, etc).

The people who say that you cannot low-level format drives now are telling
the truth. To actually rewrite the *real* low level stuff, like sector
placements, etc, you will need to contact the manufacturer. These real
low-level formats are vendor- (and often model-) specific. So you can't do a
low-level format. The best you can do is a medium-level format, which is
what the "low-level format" option in your BIOS screen does (if you have
one). It is easy to get the drive information from the controller in a
standard format (command ECh), and from this the BIOS can do a format as
above. This is as low as you can get. Some manufactures do release real
low-level format utilities but I have not got around to reversing them to
see what they actually do. Feel free to try :)

Quote:
> - ...
> I have other doubts, but first of all I want to know if there is
> anybody that really can help me.
> Thanks!

Probably the IDE/ATA spec could help you a lot here :)

--
Michael



Tue, 01 Feb 2005 14:55:56 GMT  
 Low level hard disk programming
Thanks Tim, is nice to see that there is somebody that can help me.
Quote:
> But you don't believe them?  What would it take to convince you?

I believe with proofs or reference to the proofs (source code in this
case, because I am supposing that my question is read by programmers,
not by casual PC users).

Quote:
> Look, if you don't believe us, go disassemble the damn BIOS yourself.  It
> isn't that hard.  You'll find that the controller in an IDE drive does a
> lot of the work independently.
> You could also go download a Linux kernel and read through the disk driver
> code.

These are reference to the proofs, you must be sure that I will do it.

Quote:
> There are spare sectors on each track of an IDE disk.  At the factory, they
> scan the disk looking for physical bad blocks.  If bad blocks are found,
> there is a table on the track that tells it which sectors are bad, and the
> drive substitutes some of the extra sectors instead.  The microcode that
> controls all of this is usually stored in a special track on the hard disk,
> and is loaded into the drive's controller upon spin-up.
> Unless you rewrite the controlware, you can't dink with the physical bad
> block table.

All of this is very interesting.
I thank if you or anybody else can give me more information about it.
I would like to see more reference of this theme
(documents,links,source code,etc).


Wed, 02 Feb 2005 06:06:40 GMT  
 Low level hard disk programming

Quote:

> I believe with proofs or reference to the proofs (source code in this

    You want proof? Low level format your own hard drive :)

See ya!
Orlando



Wed, 02 Feb 2005 07:56:14 GMT  
 Low level hard disk programming
Michael: thank you very much. This is the answer I was waiting for. I
am searching info about all of this, but I want to begin to program
something now. First of all I am interested in knowing how I can do
the bad sector marking that you name in:

Quote:
> The second is putting a "bad sector" mark on the sector. This does not fix
> it, but just writes a byte to the sector data that when read back tells the
> HDD to report it as a bad sector. This effectively does nothing apart from
> provide one bit of storage per sector, for you to use in whatever way you
> want (however it should be used for bad sector marking).

Also, I want to know if I can mail you ocasionally for other questions
of this type, because as you can see here there are not real
programmers that can help me.


Sun, 06 Feb 2005 08:56:03 GMT  
 Low level hard disk programming
On Fri, 16 Aug 2002 06:55:56 +0000 (UTC), "Michael Brown"

<snip>

Quote:
>The second is putting a "bad sector" mark on the sector. This does not fix
>it, but just writes a byte to the sector data that when read back tells the
>HDD to report it as a bad sector. This effectively does nothing apart from
>provide one bit of storage per sector, for you to use in whatever way you
>want (however it should be used for bad sector marking).

I wouldn't have thought that this would work for any HDD later than an
MFM or RLL drive, and possibily early IDE drives.  I do remember doing
this type of thing back about 20 years ago or so, but not for PC type
drives.

<snip>
--
Arargh (at arargh dot com)    http://www.arargh.com
To reply by email, change the bogus reply address.



Sun, 06 Feb 2005 11:58:56 GMT  
 Low level hard disk programming

Quote:
> Michael: thank you very much. This is the answer I was waiting for. I
> am searching info about all of this, but I want to begin to program
> something now. First of all I am interested in knowing how I can do
> the bad sector marking that you name in:

> > The second is putting a "bad sector" mark on the sector. This does not
fix
> > it, but just writes a byte to the sector data that when read back tells
the
> > HDD to report it as a bad sector. This effectively does nothing apart
from
> > provide one bit of storage per sector, for you to use in whatever way
you
> > want (however it should be used for bad sector marking).

This is done through the formatting as described before. For each sector you
want a "bad" mark in, set the relevant byte of the sector data table to 80h,
to mark it as "bad". Then, when reading the data back (ATA/IDE command 20h),
check the value of port 1F1h (hard disk error register) for each sector to
see if there was an error. If the "bad sector" bit has been set for the
sector, then bit 7 of the error register will be set. It is good practice to
read this register anyhow, as it reports stuff like ECC errors etc. I'm not
sure how this all works when using DMA transfers however.

Quote:
> Also, I want to know if I can mail you ocasionally for other questions
> of this type,

Sure no problem. I've updated my signature to let people know this now :)

Quote:
> because as you can see here there are not real
> programmers that can help me.

There are a *lot* of good programmers on this group: Orlando Llanes and
Randall Hyde spring immediately to mind as they are in my address list :)
However, this is not the best place for low-level stuff like this, probably
as OS development group might be able to help more.

--
Michael Brown
My inbox is always open (remove the obvious):



Sun, 06 Feb 2005 15:56:00 GMT  
 Low level hard disk programming
""Arargh (CircleAroundAnA NOSPAMarargh decimal NOSPAMcom)" "

Quote:
> On Fri, 16 Aug 2002 06:55:56 +0000 (UTC), "Michael Brown"

> <snip>
> >The second is putting a "bad sector" mark on the sector. This does not
fix
> >it, but just writes a byte to the sector data that when read back tells
the
> >HDD to report it as a bad sector. This effectively does nothing apart
from
> >provide one bit of storage per sector, for you to use in whatever way you
> >want (however it should be used for bad sector marking).

> I wouldn't have thought that this would work for any HDD later than an
> MFM or RLL drive, and possibily early IDE drives.  I do remember doing
> this type of thing back about 20 years ago or so, but not for PC type
> drives.

Yup, works fine on a 2 gig Quantum Fireball (yeah, I know, this is stone-age
stuff now ... ). I haven't tried it on my 20 gig HDD because I have stuff on
it I don't want to lose :)

Format Track was in the ATA-1 (aka IDE) standard, was marked as
"optional/vendor specific" (along with the note "It is recommended that
system implementations not utilize this command." in ATA-2 and ATA-3 (both
also called EIDE), and was gone from ATA/ATAPI-4 (aka UDMA-33), in the form
of being marked obsolete. This is all from the specs at
http://www.t13.org/

Another good description of IDE here:
http://www.gaby.de/gide/IDE-TCJ.txt

I would guess that most older (ATA-3 and before) drives would support it,
quite possibly just emulating it though and not actually doing any
formatting. I have no idea about ATA/ATAPI-4 and later.

--
Michael Brown
My inbox is always open (remove the obvious):



Sun, 06 Feb 2005 16:55:58 GMT  
 Low level hard disk programming

Quote:

> There are a *lot* of good programmers on this group: Orlando Llanes and
> Randall Hyde spring immediately to mind as they are in my address list :)

    :)

    kkarma's not worth any of our flaming time. According to him, if you
don't know how to low level format a hard drive with assembly language then
you don't know crepe. Pardon my French ;)
    For the annoyed, I promise that this is my final post on the subject.

See ya!
Orlando



Mon, 07 Feb 2005 01:56:09 GMT  
 Low level hard disk programming


Quote:
>I looked for information about hard disk's low level format on various
>groups (eg. comp.lang.asm.x86,comp.os.linux), but my search only took
>me to posts like: "don4t do it", "it's hardware vendor specific", "the
>new ide doesn4t make a real low level format".
>Now, I will make a few questions that involves low level programming
>and I like to see professional answers, not the simple answers like
>the above mentioned.
>Then, my questions are:
>- How is implemented the Int13,function 5 actually for hard disks?

This is not fully done on new systems because the IDE interface
does not use this information anymore.

The modern drives of the IDE/ATA variety actually use the
LBA absolute sector number for everything, No matter how you
address the drive.

To do the format like you are talking about you must first tell
the drive to use your CHS cylinder, head, sector values.

Then you just go ahead and find the appropriate commands
in the list IDE/ATA low level stuff and send it.

Quote:
>- How is implemented the bad sector marking on a hard disk? (not the
>FFF7h cluster mark in the FAT), but the real table management that
>knows which physical sector is really bad, something like the "track
>address fields" for a floppy disk format.

These are generically known as P&G tables. Permanent and gathered
errors table or list.

This idea comes from SCSI which is where much of the modern
improvments in IDE have come from. Approximatly 3% of the
drive on a physical track by physical track basis is set aside to
catch gathered errors as they occur.

In order to fully under stand this look at a modern drive size.

162000 * 16 * 63 done on an actual 2 platter or 4 Head system. (~80-Gig)

this means that you actually deal with the following.

648000 * 4 *63 is the first step.

Then you factor in the 6000Tpi of tracks per inch density.
the track are the same as cylinders (concentric rings goinf outward.)
You actually have about an inch of track width on a 3 1/2 drv.
so you actually get something like.

6000 * 4 * 6804 is where you end up.

Now if it was me I would just use the 6144 (1024 * 6)
as my SPT (Sectors per track) or about 3Megs per physical
track on the media.

This illustrate the departure of the IDE/ATA technology from
it's FM / MFM / RLL / ESDI roots. The sector address no longer
represented the physical address of the sector.
This now represented a virtual position and could be changed
at will by the onboard electronics for data preservation.
All commands to access the P&G tables are vendor specific.

Quote:
>- How does the BIOS low level format work and which are the supported
>disks (it means, vendor, capacity, etc).

Please see the above then add this to it.

Modern drives use a two layer approach to a format.
The sector boundary's are no longer variable.

Now there is an inner layer of hard coded sectors
that the onboard electronics can only read nor write.
Then there is the soft coded parts that contain sector
specific data. This can be CHS placement Information
in addition to LBA address & sector data.

Quote:
>- ...
>I have other doubts, but first of all I want to know if there is
>anybody that really can help me.

If you really want to get into the nitty gritty then I suggest
you grab the BT168.zip off of my site and study it closely.

Messy though the code is, poorly documented it may be.
How ever it runs spinrite.
It also handles upto 128 Gig drives. in CHS upto 8-Gig and
in LBA from there up.

Done myself from the ground up.
When I get time I will be cleaning up the code and makeing some
real documemtation for it.

Quote:
>Thanks!

walk.to/doors

--
Doors - Dont look at the future in a window.
        just walk to the door http://walk.to/doors
      - Open it and go there.

http://www.humboldt1.com/~doors/doors.htm
OR SEARCH AT
http://www.humboldt1.com/~doors/search.htm





Sat, 19 Feb 2005 01:19:23 GMT  
 
 [ 11 post ] 

 Relevant Pages 

1. Low level hard disk access

2. Floppy Disk Drive Low Level Programming Help!

3. Access disk Low level ?!

4. Low level disk services - Usual newbie needs help type of message

5. Low-level disk read/write

6. Low-level disk control

7. Code to Low-Level read/write to Floppy Disks

8. Low level disk access

9. Low level disk/file info from 32bit VC?

10. Low level floppy disk stuff

11. Really Low Low Low Level Stuff.

12. Low-level disk i/o in DOS

 

 
Powered by phpBB® Forum Software