Skip to content

01 Data Storage

Jump to bottom
Jordy Homing Lam edited this page Dec 12, 2021 · 12 revisions

Semi-Permanent Storage Principle

My laptop, like many other recent laptops in the market, is only equipped with a Solid-State-Drive (SSD). There is no Hard-Disk-Drive (HDD) for permanent storage of data. Unlike an HDD, an SSD can only endure a limited number of write/erase cycles also known as program/erase (P/E) cycle. This is because the oxide layer that traps electrons in a NAND flash memory cell will deteriorate when a P/E cycle is completed. In other words, the SSD will eventually become unreliable - it will wear out and ultimately lose its ability to write data, even though there are mechanisms to prevent unsuccessful read after the SSD gone completely worn out. A few implications to ML tasks.

  • SSD in general offers faster access to large-volume data compared to HDD. This is a big plus.
  • SSD is also more shock-resistant compare to a spinning disk drive. Good for a laptop to be used on a bumpy road.
  • However, we should not frequently update (erase and write) data/model into an SSD. We should reduce the number of P/E cycles. If we observe this rule, we have a nice almost-permanent storage.

Creating Temporary Storage

Wait. I have a habit of downloading programs and arxiv papers with nice graphics and making slides for conference and hundreds of revisions of manuscript. My storage demand grows linearly. How to deal with this? A solution is buy oneself a thumb drive 500 GB and make all temporary storage there. This include many default directories

A nice tiny thumb drive (pinky drive!) will be SanDisk 512GB Ultra Fit USB 3.1 Flash Drive - SDCZ430-512G-G46. This can be plugged into the USB slot 24/7. You should expect this drive to die anytime, so never store anything important there.

How about documents, slides, etc? These has to be stored somewhere safer, right? I would suggest Microsoft's cloud drive.

Clone this wiki locally