Slow format ?

[Abandoned Trolley]

I am guessing that when formatting disk drives, whether they are
spinning rust, SSD or USB memory sticks ... most people check the "quick
Format" box and let the machine get on with it (Windows 11 in this case)


Just out of curiosity, I put a 2TB USB 3.0 memory stick in my machine
yesterday, and asked it to do a NTFS "non-quick" format.

Its now been going for about 18 hours - and its somewhere between a
quarter and a third of the way through. The little bar graph thing is
updating, so I am fairly sure its not actually frozen.

Is this anything remotely like normal ? - or do people simply never do
this ?

[GB]

Post by Abandoned Trolley
I am guessing that when formatting disk drives, whether they are
spinning rust, SSD or USB memory sticks ... most people check the "quick
Format" box and let the machine get on with it (Windows 11 in this case)
Just out of curiosity, I put a 2TB USB 3.0 memory stick in my machine
yesterday, and asked it to do a NTFS "non-quick" format.
Its now been going for about 18 hours - and its somewhere between a
quarter and a third of the way through. The little bar graph thing is
updating, so I am fairly sure its not actually frozen.
Is this anything remotely like normal ? - or do people simply never do
this ?

It sounds a bit slow, but not impossible. Supposing the write speed is
20 Mbps, how many hours is that to fill up 2TB? I'll leave you to do
the sums.

[Andy Burns]

I put a 2TB USB 3.0 memory stick in my machine yesterday, and asked it
to do a NTFS "non-quick" format.
Its now been going for about 18 hours - and its somewhere between a
quarter and a third of the way through. The little bar graph thing is
updating, so I am fairly sure its not actually frozen.
Is this anything remotely like normal ?

Yep

or do people simply never do this ?

Well, maybe only once to see how long it takes ...

[Jeff Gaines]

I am guessing that when formatting disk drives, whether they are spinning
rust, SSD or USB memory sticks ... most people check the "quick Format"
box and let the machine get on with it (Windows 11 in this case)
Just out of curiosity, I put a 2TB USB 3.0 memory stick in my machine
yesterday, and asked it to do a NTFS "non-quick" format.
Its now been going for about 18 hours - and its somewhere between a
quarter and a third of the way through. The little bar graph thing is
updating, so I am fairly sure its not actually frozen.
Is this anything remotely like normal ? - or do people simply never do
this ?

Wasn't it the only option for a Windows install once upon a time?

[Abandoned Trolley]

Post by Jeff Gaines

Post by Abandoned Trolley
I am guessing that when formatting disk drives, whether they are
spinning rust, SSD or USB memory sticks ... most people check the
"quick Format" box and let the machine get on with it (Windows 11 in
this case)
Just out of curiosity, I put a 2TB USB 3.0 memory stick in my machine
yesterday, and asked it to do a NTFS "non-quick" format.
Its now been going for about 18 hours - and its somewhere between a
quarter and a third of the way through. The little bar graph thing is
updating, so I am fairly sure its not actually frozen.
Is this anything remotely like normal ? - or do people simply never do
this ?

Wasn't it the only option for a Windows install once upon a time?

I believe so - but ISTR that most Windows systems had a volume limit of
2GB at the time ?

[Philip Herlihy]

Post by Abandoned Trolley
I am guessing that when formatting disk drives, whether they are
spinning rust, SSD or USB memory sticks ... most people check the "quick
Format" box and let the machine get on with it (Windows 11 in this case)

I'll usually do a full format of a new disk. As I understand it, a quick
format simply installs the filesystem (most of it empty). A full format also
does a surface integrity check, which might involve (hidden) re-mapping of any
bad sectors which are found. (When re-mapping becomes visible, that means the
'spare' sectors are used up, and the disk is on the way out.) When in use,
I'll normaly use a SMART monitor (currently HDTune) to watch out for
deterioration, and that has saved my bacon (that's data-bacon) many times over
the years.

[Philip Herlihy]

In article <***@news.eternal-september.org>, Philip
Herlihy wrote...

Post by Philip Herlihy
I'll normaly use a SMART monitor (currently HDTune)

Typo - HDTune is a free utility which *can* read SMART data. I actually use
HDSentinel, which warns you proactively of any deterioration (recommended).

[Theo]

Post by Philip Herlihy

Post by Abandoned Trolley
I am guessing that when formatting disk drives, whether they are
spinning rust, SSD or USB memory sticks ... most people check the "quick
Format" box and let the machine get on with it (Windows 11 in this case)

I'll usually do a full format of a new disk. As I understand it, a quick
format simply installs the filesystem (most of it empty). A full format also
does a surface integrity check, which might involve (hidden) re-mapping of any
bad sectors which are found.

That dialogue is very old. Originally 'format' meant lay down disk
sectoring information, used to navigate the disk (putting the disc into a
certain format of X sectors by Y tracks), which is what happens to a
floppy. On a hard drive made later than 1980s, the sectoring information is
not exposed to the OS and is internal to the drive. Thus 'format' hasn't
done actual formatting for decades.

There are two modes that have been relevant on HDD for the last 35 years:

1. write just the partition information, leaving most data blocks holding
old contents
2. write the partition information and also wipe all blocks on the disc

Windows calls #1 'quick format' and #2 'format', 1980s terminology.

With modern SSDs there is no point doing #2, because a simple change of the
encryption key (which all SSDs have internally) will scramble the contents
of the disc so all private data is unreadable[*].

[*] I'm not exactly sure how Windows implements that key changing process,
but I think BIOS wiping the disc does it

So you'd only really do the full data write if there was something wrong
with the disc and you wanted to try to reuse it by clearing bad blocks, or
perhaps if you wanted to sell it and to be extra sure it was wiped.

For SSD it is actually counterproductive for a new drive because it it eats
one of your limited flash write cycles. A random 1TB SSD has 240TBW, meaning
you can write the whole drive 240 times before it dies. If you do a full
format you now only have 239.

Theo

[Abandoned Trolley]

Post by Theo
For SSD it is actually counterproductive for a new drive because it it eats
one of your limited flash write cycles. A random 1TB SSD has 240TBW, meaning
you can write the whole drive 240 times before it dies. If you do a full
format you now only have 239.
Theo

so ... the regular reading and writing of swap files in virtual memory
systems is destroying SSDs all over the place ?

[Andy Burns]

so  ... the regular reading and writing of swap files in virtual memory
systems is destroying SSDs all over the place ?

reading - not so much, it's the writing that hurts, these days just buy
RAM to avoid swapping if the system ca

[Theo]

Post by Abandoned Trolley

Post by Theo
For SSD it is actually counterproductive for a new drive because it it eats
one of your limited flash write cycles. A random 1TB SSD has 240TBW, meaning
you can write the whole drive 240 times before it dies. If you do a full
format you now only have 239.
Theo

so ... the regular reading and writing of swap files in virtual memory
systems is destroying SSDs all over the place ?

Yes, writing is causing wear. But if the SSD has 240TBW and your device has
8GB RAM, you'd need to write the whole RAM contents 30,000 times. If you
are swapping 1GB it would be 240,000 times, etc. So it depends how much you
are writing out and how frequently. Bearing in mind that for many SSDs
writing 1GB would take a full second or thereabouts, most swapping is going
to be much smaller writes.

For those kind of machines one option is just to buy SSDs with more write
cycles. Cheap consumer SSDs have a lot fewer cycles than more enterprise-y
ones.

But better to fit more RAM if you can.

Theo

[David]

Post by Theo

Post by Abandoned Trolley

Post by Theo
For SSD it is actually counterproductive for a new drive because it
it eats one of your limited flash write cycles. A random 1TB SSD has
240TBW, meaning you can write the whole drive 240 times before it
dies. If you do a full format you now only have 239.
Theo

so ... the regular reading and writing of swap files in virtual memory
systems is destroying SSDs all over the place ?

Yes, writing is causing wear. But if the SSD has 240TBW and your device
has 8GB RAM, you'd need to write the whole RAM contents 30,000 times.
If you are swapping 1GB it would be 240,000 times, etc. So it depends
how much you are writing out and how frequently. Bearing in mind that
for many SSDs writing 1GB would take a full second or thereabouts, most
swapping is going to be much smaller writes.
For those kind of machines one option is just to buy SSDs with more
write cycles. Cheap consumer SSDs have a lot fewer cycles than more
enterprise-y ones.
But better to fit more RAM if you can.
Theo

Just checking my understanding.

Back in the spinning rust days you would allocated a small partition as
SWAP.
You might even position it on the disc to have the potential fastest
access time for seeks.
With SSDs blocks are allocated on a virtual rather than physical layout so
a 10 GB SWAP space will not be continuously reusing contiguous blocks over
a small area of the SSD.

Yes to more RAM.

I think.

Cheers



Dave R

[Abandoned Trolley]

Post by David
Back in the spinning rust days you would allocated a small partition as
SWAP.
You might even position it on the disc to have the potential fastest
access time for seeks.
With SSDs blocks are allocated on a virtual rather than physical layout so
a 10 GB SWAP space will not be continuously reusing contiguous blocks over
a small area of the SSD.
Yes to more RAM.
I think.
Cheers
Dave R

I have done a number of Solaris installs on SPARC systems, and tailored
the swap partition according to the memory - but I cant remember setting
up any swap partition on any Windows system.

I dont think too many people mess around with that sort of thing - but I
guess theres nothing preventing it on spinning rust systems

[David]

Post by Abandoned Trolley

Post by David
Back in the spinning rust days you would allocated a small partition as
SWAP.
You might even position it on the disc to have the potential fastest
access time for seeks.
With SSDs blocks are allocated on a virtual rather than physical layout
so a 10 GB SWAP space will not be continuously reusing contiguous
blocks over a small area of the SSD.
Yes to more RAM.
I think.

I have done a number of Solaris installs on SPARC systems, and tailored
the swap partition according to the memory - but I cant remember setting
up any swap partition on any Windows system.
I dont think too many people mess around with that sort of thing - but I
guess theres nothing preventing it on spinning rust systems

Having come from a background where disc tuning was important (including
SPARC) I used to do this on Windows.
Create a specific SWAP partition on the disc.
Sometimes create SWAP partitions on multiple discs to spread the load.

All this is behind me now.
I just close my eyes and let Windows get on with it.

Cheers



Dave R

[P.S. remember going up from a 10 MB disc to a 20 MB disc and ruling the
world?]

[Jaimie Vandenbergh]

Post by David
With SSDs blocks are allocated on a virtual rather than physical layout so
a 10 GB SWAP space will not be continuously reusing contiguous blocks over
a small area of the SSD.

That's correct, it won't.

But on a flash drive or SD, it might :) They have dumber controllers.

Cheers - Jaimie

[Andy Burns]

Post by David
With SSDs blocks are allocated on a virtual rather than physical layout so
a 10 GB SWAP space will not be continuously reusing contiguous blocks over
a small area of the SSD.

The contiguous (or otherwise) allocation of blocks doesn't matter for an
SSD or NVMe device, its controller will have wear-levelling that ensures
the writes get fairly shared across all block on the "disk"

[Jaimie Vandenbergh]

On 19 Apr 2024 at 07:59:01 BST, "Abandoned Trolley"

Post by Abandoned Trolley

Post by Theo
For SSD it is actually counterproductive for a new drive because it it eats
one of your limited flash write cycles. A random 1TB SSD has 240TBW, meaning
you can write the whole drive 240 times before it dies. If you do a full
format you now only have 239.
Theo

so ... the regular reading and writing of swap files in virtual memory
systems is destroying SSDs all over the place ?

Not enough to care about. The fear over limited write cycles was with
the earliest tiny SSDs (like 20 or 60gig) with lower stability storage.

These days eg a samsung 980 1TB is warrantied to 600TB of writes which
is an awful lot of full formats at 60 hours each! Four years of them,
and the warranty time limit on those 980s is five years... And that's a
consumer grade SSD, not a fancy one.

In normal use you'll write a few gig per day perhaps, and that 600TB
total will outlast many spinning disks. At 50gig/day that's 30 years.

Cheers - Jaimie