Sign up here and you can log into the forum!

What's the problem with GPT (>2 TB) support?

Have a question about devices internals, memory layout, reverse engineering, etc---This is the place for anything so technical that it would cause a n00b's head to 'splode

Re: What's the problem with GPT (>2 TB) support?   

Postby lovilela » Thu Aug 13, 2015 7:33 am

Ok, agree. I'll dump 2TB+ worth of data in my clean 5TB drive and plug it in to see what happens. Based on what you said, it should fail.
Should the fail confirms, I will format the thing in GPT+EXT3 (I read somewhere that EXT4 does not work on WDTV Live Plus) and try again.

The only challenge is that my computer is quite old, i.e. does not have USB3.0 ports, so copying data to my new drive was reaching about 20-22MB/s only on Windows 7 explorer. That means something around 40 hours to populate it with around 3TB.... :shock:

Maybe there is a more efficient way, i'll google for something.
lovilela
n00b
 
Posts: 17
Joined: Wed Aug 12, 2015 6:01 pm

Re: What's the problem with GPT (>2 TB) support?   

Postby mad_ady » Thu Aug 13, 2015 10:58 am

Two problems - the corruption might be visible in data over the 2TB mark, but if you copy all data over, how will you know what files are after the 2TB mark?

For faster file transfers try dd:
Code: Select all
time dd if=/dev/zero of=/mnt/disk/large-file-100G bs=1M count=100000

Just don't expect to play it back. Should be enough to fill 2TB and then copy some regular files to check integrity.
User avatar
mad_ady
Developer
 
Posts: 4562
Joined: Fri Nov 05, 2010 9:08 am
Location: Bucharest, Romania

Re: What's the problem with GPT (>2 TB) support?   

Postby lovilela » Thu Aug 13, 2015 12:56 pm

Good point. So, the best approach, I guess, is do some incremental testing:

1st) copy some movies (few GB only) in the empty drive, so these stay in the "beginning". Test it.

2nd) generate around 3TB worth of dummy files (certainly faster than copying stuff into the drive). Plug to see if the drive gets recognized. If yes, test the very same movies sitting on the beginning, to check whether they are still readable.

3rd) assuming that all goes well so far, copy some other movies to the drive. These new movies should be sitting then far after the 2TB mark. Test it again and try to read all movies.

I see that you recommended a command line to generate a dummy file in Linux, but i am not too familiarized with it. I found some windows tools that serve the same purpose, i.e. generate dummy files sizing 3TB or more in total. That should be lot quicker than actually moving real data to the drive, which would take 2 days on USB2.0.

I'll try this tonight and post here. By the way, the 1st step above was already done successfully.

Let me know if the sequence above has any flaw. Thanks!
lovilela
n00b
 
Posts: 17
Joined: Wed Aug 12, 2015 6:01 pm

Re: What's the problem with GPT (>2 TB) support?   

Postby lovilela » Thu Aug 13, 2015 3:40 pm

2nd part of test completed, successfully :D . I generated some dummy files on the 5TB drive, using fsutil command under windows.

Code: Select all
fsutil file createnew e:\temp\tempfile.txt 1000000000


I populated around 3.5TB in files and plugged it in on WDTV. No problems whatsoever. WDLXTV's Web Frontend shows accurately the drive size.

Image

Then I ran some of the video files placed in the beginning of the hard drive. No problems.

Now I'll run the final test, copying some video files that should be sitting now after the 2TB mark. Let's see....
lovilela
n00b
 
Posts: 17
Joined: Wed Aug 12, 2015 6:01 pm

Re: What's the problem with GPT (>2 TB) support?   

Postby lovilela » Thu Aug 13, 2015 4:20 pm

3rd part of test completed. It's working!!! :lol:

Recap: I copied some movies to the drive 3.5TB full. Plugged it on WDTV and went to the Video section. All files and movie folders were there. Hit play and no issues.

Now i'm kind of puzzled (i guess you are too). All these reports stating that 2TB+ drives wouldn't work on WDLXTV-PLUS, but really this one seems to work straight from the box, factory formatted.

Is this drive different in some way? Maybe... Look also what I found:
http://wdtvhd.com/index.php?showtopic=32561&st=0&p=84755&#entry84755

The guy does not make clear whether he was using stock or wdlxtv firmware, but he says that a 4TB Seagate Backup Plus is working just fine.

Anyway, I will now format the drive to GPT+EXT3, copy all my stuff and hook it into WDTV. 8-)
lovilela
n00b
 
Posts: 17
Joined: Wed Aug 12, 2015 6:01 pm

Re: What's the problem with GPT (>2 TB) support?   

Postby mad_ady » Thu Aug 13, 2015 8:01 pm

Good job!

Before you format the drive, can you gather this output?
Code: Select all
uname -a
cat /conf_src/version
df -h
fdisk -l
User avatar
mad_ady
Developer
 
Posts: 4562
Joined: Fri Nov 05, 2010 9:08 am
Location: Bucharest, Romania

Re: What's the problem with GPT (>2 TB) support?   

Postby lovilela » Fri Aug 14, 2015 2:19 pm

Ops, just formatted... but I think the question that entitles the topic is the answer on why this drive just worked out of the box. I burned a Ubuntu live CD to use GParted and format it on EXT3 and found that the drive comes factory-partitioned in GPT, instead of the usual MBR. It shows two partitions, one sizing 128MB (the MSR) and another with 4.5TB in NTFS. The only strange thing is that GParted displays an error when starting, saying that the "GPT backup partition is corrupt, using primary instead". Then it opens... I googled it and some says the solution is to use gdisk and write again the GPT tables, but that did nothing, the message remains when I open GParted. Anyway, from the info in gdisk, it seems that the drive is ok (and it should, it's brand new and factory formatted!) and I decided to go ahead and format the big partition in EXT3 (took several hours to finish), leaving the MSR untouched.

Anyway, I'll run the commands you asked and advise here. After all, the partitioning was kept, i just formatted the NTFS one to EXT3, that's all.
lovilela
n00b
 
Posts: 17
Joined: Wed Aug 12, 2015 6:01 pm

Re: What's the problem with GPT (>2 TB) support?   

Postby lovilela » Fri Aug 14, 2015 7:07 pm

find below the command sequence. The Seagate 5TB partitions are sda1 and sda2:

Code: Select all
ubuntu@ubuntu:~$ uname -a
Linux ubuntu 3.16.0-23-generic #31-Ubuntu SMP Tue Oct 21 18:00:35 UTC 2014 i686 i686 i686 GNU/Linux
ubuntu@ubuntu:~$ cat /conf_src/version
cat: /conf_src/version: No such file or directory
ubuntu@ubuntu:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/cow            1.5G   37M  1.5G   3% /
udev            1.5G  4.0K  1.5G   1% /dev
tmpfs           303M  1.2M  302M   1% /run
/dev/sr0        1.2G  1.2G     0 100% /cdrom
/dev/loop0      1.1G  1.1G     0 100% /rofs
none            4.0K     0  4.0K   0% /sys/fs/cgroup
tmpfs           1.5G  1.1M  1.5G   1% /tmp
none            5.0M  8.0K  5.0M   1% /run/lock
none            1.5G  232K  1.5G   1% /run/shm
none            100M   60K  100M   1% /run/user
/dev/sdc1       1.9T  1.8T  102G  95% /media/ubuntu/My Passport
/dev/sda2       4.6T   64M  4.3T   1% /media/ubuntu/Seagate
ubuntu@ubuntu:~$ sudo fdisk -l

Disk /dev/loop0: 1.1 GiB, 1147199488 bytes, 2240624 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sda: 4.6 TiB, 5000981077504 bytes, 9767541167 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 33553920 bytes
Disklabel type: gpt
Disk identifier: FF6EAFDE-0B03-440B-994B-4D1FDF1B4508

Device      Start        End    Sectors  Size Type
/dev/sda1      34     262177     262144  128M Microsoft reserved
/dev/sda2  264192 9767540735 9767276544  4.6T Microsoft basic data

Partition 2 does not start on physical sector boundary.


Disk /dev/sdb: 111.8 GiB, 120034123776 bytes, 234441648 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xc5467b6f

Device     Boot  Start       End   Sectors   Size Id Type
/dev/sdb1  *      2048    206847    204800   100M  7 HPFS/NTFS/exFAT
/dev/sdb2       206848 234438655 234231808 111.7G  7 HPFS/NTFS/exFAT

Disk /dev/sdc: 1.8 TiB, 2000365289472 bytes, 3906963456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0005f107

Device     Boot Start        End    Sectors  Size Id Type
/dev/sdc1        2048 3906963455 3906961408  1.8T  7 HPFS/NTFS/exFAT

ubuntu@ubuntu:~$
lovilela
n00b
 
Posts: 17
Joined: Wed Aug 12, 2015 6:01 pm

Re: What's the problem with GPT (>2 TB) support?   

Postby mad_ady » Fri Aug 14, 2015 10:20 pm

Ok, the output looks good. Is it taken after formatting to ext3? If yes, be on the lookout for issues - the partition is marked as microsoft data in gpt. Shouldn't matter for the wdtv, but might confuse windows if you plug it in.
User avatar
mad_ady
Developer
 
Posts: 4562
Joined: Fri Nov 05, 2010 9:08 am
Location: Bucharest, Romania

Re: What's the problem with GPT (>2 TB) support?   

Postby lovilela » Sat Aug 15, 2015 9:12 am

Yes, it was taken after formatting, but in Ubuntu. Now a report on a probable bug on 0.5.2.2 related to this 2TB limit.

Yesterday, after formatting the 5TB drive, I hooked it on WDTV, copied a movie to test (it was ok) and then I thought: now it is time to upgrade WDLXTV from 0.5.1.1 to 0.5.2.2 and get the benefit of rtorrent 0.9.2, plus other bugs fixed. The 2TB limit issue for me seemed to be over, GPT was the solution.

So, I flashed 0.5.2.2 and when started, rtorrent was not starting. Initially I thought it was something specific, maybe a missing configuration. I read almost everything here on rtorrent and still no success. I was puzzled again... just when a 5TB works rtorrent now is broken!?

But then I noticed something very strange... in the web frontend the drive was showing the right size (4.5TB) but the format displayed was EXT2, instead of EXT3, the actual file format. I decided to access WDTV using SSH and ran the fdisk -l command and to my surprise, the drive partition was showing as 2.1TB! Now the reports on that bug started to make sense. I guessed I may overlooked something and indeed all the time maybe fdisk -l would be showing 2.1TB size, and indeed no way I could use my brand new 5TB on WDLXTV.

But before giving up I decided to give a last try, because I remembered the web frontend showing accurately the partition type EXT3 on 0.5.1.1. So I downgraded WDTV Plus to 0.5.1.1 and the info was ok again (showing EXT3). Through SSH I ran fdisk -l and the result showed 5TB, so I think there is something between the 0.5.1.1 and 0.5.2.2 that is actually broking.

Here are the results from fdisk -l on 0.5.1.1.

Code: Select all
login as: root
root@192.168.1.40's password:
   .-.-.-..--. .-.   .-..-..---..-..-.
   | | | || \ \| |__  >  < `| |' \  /
   `-----'`-'-'`----''-'`-` `-'   `'
       1.05.04_B_WDLXTV.COM_WDLXTV_PLUS-0.5.1.1
# fdisk -l

Disk /dev/sigmblocka: 267 MB, 267911168 bytes
255 heads, 63 sectors/track, 32 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblocka doesn't contain a valid partition table

Disk /dev/sigmblockb: 0 MB, 524288 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockb doesn't contain a valid partition table

Disk /dev/sigmblockc: 0 MB, 262144 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockc doesn't contain a valid partition table

Disk /dev/sigmblockd: 3 MB, 3145728 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockd doesn't contain a valid partition table

Disk /dev/sigmblocke: 3 MB, 3145728 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblocke doesn't contain a valid partition table

Disk /dev/sigmblockf: 16 MB, 16777216 bytes
255 heads, 63 sectors/track, 2 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockf doesn't contain a valid partition table

Disk /dev/sigmblockg: 8 MB, 8388608 bytes
255 heads, 63 sectors/track, 1 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockg doesn't contain a valid partition table

Disk /dev/sigmblockh: 94 MB, 94371840 bytes
255 heads, 63 sectors/track, 11 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockh doesn't contain a valid partition table

Disk /dev/sigmblocki: 94 MB, 94371840 bytes
255 heads, 63 sectors/track, 11 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblocki doesn't contain a valid partition table

Disk /dev/sigmblockj: 0 MB, 131072 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockj doesn't contain a valid partition table

Disk /dev/sigmblockk: 0 MB, 131072 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockk doesn't contain a valid partition table

Disk /dev/sigmblockl: 0 MB, 131072 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sigmblockl doesn't contain a valid partition table

Disk /dev/sda: 5000.9 GB, 5000981077504 bytes
255 heads, 63 sectors/track, 608001 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sda1               1      267350  2147483647+ ee EFI GPT


So, to make a long story short, 0.5.1.1 seems to deal with 5TB GPT drives (despite b-rad's alert on the firmware release notes) and 0.5.2.2 not.

About rtorrent, it is working on 0.5.1.1, but the version is the old 0.8.6, so I will now try 0.5.2.1 version in EXT3 boot mode to see whether I can have both fdisk recognizing 5TB and rtorrent 0.9.2 running.
lovilela
n00b
 
Posts: 17
Joined: Wed Aug 12, 2015 6:01 pm

PreviousNext

Return to WDTV Live

Who is online

Users browsing this forum: No registered users and 1 guest