unpack / repack a firmware

[b-rad.cc]

This is a simple guide to how to unpack / repack a firmware.

First a little history...

On the original WDTV and other similar devices before it kernel page size was a standard 4KB. 4KB is used as kernel page size in most modern systems, and therefore standard cramfs tools worked just fine with the firmware images. This all changed however when Sigma released their 2.0 SDK, which changed the kernel page size to 16KB...only thing is they didn't publish this little factoid due to their extreme security policies. Now when the WDTV Live came out it used the 2.0 SDK...and when I got my hands on a device when it hit the market I instantly hit a roadblock trying to port over wdlxtv-live. Well luckily I don't give up very easily so I got to reverse engineering. It was really a mystery at first, because while cramfs images mounted and were read fine on the device itself--those same images were invalid off the device. Was their some sort of encryption at play here? I couldn't see how their could be, unless it applied specifically to cramfs as all other filesystem images mounted correctly on the device. So what did I do? I spent pretty much the next 100hours, straight, mapping out the cramfs file structure by hand. Talk about the hard way. What did I find? Well cramfs is a sort've dictionary based filesystem, you have all the filenames listed at the start of the filesystem and each one has an address and an offset. The address is basically the block number and the offset is where the filecontents start. Note: completely laymanizing the description. Hmmm...all still looked good and valid, like cramfs is supposed to be. So then I re-generated an exact copy of an official fw rip using the normal cramfs tools and compared it byte-by-tedious-byte trying to find where they differed. This is where I noticed something peculiar, in the new firmware images there were only a quarter of the filesystem blocks! Eureka. Once I had discovered this and re-looked at the cramfs source code it was pretty obvious that kernel page size was really the only variable option--and it even says right in the source code AND in the accompanying NOTES file that block size is equal to kernel page size. Maybe I need to learn to read before I go about things the hard way. So now all that was needed was a bit of confirmation before I became the first wdtvl rma. Well heres how I confirmed it:

Code: Select all

cat /proc/meminfo | grep Mapped
cat /proc/vmstat | grep nr_map

Divide the two and you get ~16KB page size, which coincidentally is block size used in cramfs. After that it was a piece of cake and cramfsck only required an increase in the rom buffer size to accomodate larger block size. At this point I flashed wdlxtv-live-0.1 and promptly went to sleep

Ok, I'll shut up now--just figured I might as well finally make public exactly how I figured it all out

firmware structure is as follows:

Code: Select all

[32 Byte md5sum][fw cramfs image][16 Byte signature]

the 32 Byte md5sum is the last part of the firmware generated. The signature details you can find here.

Heres the run down:

You have a wdtvlive.bin (WDTV Live) or wdtvhd.bin (WDTV G2) in front of you.

Code: Select all

tail -c +33 wdtvlive.bin > wdtvlive.stripped.bin
cramfsck-16k -x dump wdtvlive.stripped.bin

There, simple as that.

Now in the opposite direction.

Code: Select all

mkcramfs-16k dump/ newFW.bin
signFW newFW.bin signature
cat newFW.bin signature > newFWx.bin
md5sum newFWx.bin | head -c 32 > wdtvlive.bin
cat newFWx.bin >> wdtvlive.bin
rm newFW.bin newFWx.bin

Voila, ez pz lemon cheezie.

Now if you're rebuilding a firmware and have modified files do not forget to update the modified files md5sums in /md5sum.txt.

Attached are the binaries & source files modified to 16k block size. Compile the source on your own to produce binaries or use the actual 32bit binaries I use, its your choice.

[simplytb]

Awesome investigation job!!!
Thanks for sharing.

SimplyTB

[juliojs]

Thank you!

I'll re-check my repack process...

[murlock]

Please note that a clone of WD TV HD requires to update /conf_src/version otherwise after flashing custom firmware, boot on it doesn't occurs.

[b-rad.cc]

its not required to modify /conf_src/version at all.

[tjay]

thanks for sharing brad!

and now a little script that handles all the steps

Code: Select all

wdtv:~# ./wdtvfw_tool
usage: ./wdtvfw_tool <pack|unpack> <firmware folder> <firmware>
wdtv:~# wdtvfw_tool unpack ./rootfs ./wdtvlive.bin
stripping firmware ...
unpack cramfs image ...
unpacked to ./rootfs
wdtv:~# wdtvfw_tool pack ./rootfs ./wdtvlive.new.bin
create cramfs image ...
sign image ...
./wdtvlive.new.bin created
wdtv:~# md5sum *.bin
61b5b33ac11a04e92635d823f0ffcf19  wdtvlive.bin
61b5b33ac11a04e92635d823f0ffcf19  wdtvlive.new.bin


with "md5sum.txt"-creation you get different final md5sums so I left this step out for demonstration

wdtvfw_tool:

Code: Select all

#!/bin/sh

cram_missing () { echo -e "please make mkcramfs-16k and cramfsck-16k available to your PATH\ndownload it here http://forum.wdlxtv.com/viewtopic.php?f=43&t=537" && quit; }
usage () { echo "usage: $0 <pack|unpack> <firmware folder> <firmware>" && quit; }
quit () { rm -f $tmp && exit 1; }

tmp=`tempfile`
type -P cramfsck-16k &>/dev/null || cram_missing
type -P mkcramfs-16k &>/dev/null || cram_missing


[ "$1" == "" ] || [ "$2" == "" ] || [ "$3" == "" ] && usage

if [ $1 == "pack" ]; then

  #pack

  [ -e $3 ] && echo "firmware \"$3\" already exists" && quit
  [ ! -d $2 ] && echo "firmware folder \"$3\" does not exist" && quit

  echo "create md5sum.txt ..."
  cwd=`pwd` && cd $2 && find  -not -name "md5sum.txt" -type f -exec md5sum '{}' ';' > md5sum.txt && cd $cwd

  echo "create cramfs image ..."
  mkcramfs-16k $2 $tmp > /dev/null || quit

  echo "sign image ..."
  filesize=`printf %x \`stat -c %s $tmp\``
  [ ${#filesize} -eq 7 ] && filesize="0$filesize"
  sig=""
  for i in 6 4 2 0 ; do
    sig="$sig\x${filesize:$i:2}"
  done
  sig="$sig\x00\x00\x00\x00"

  echo -en "\xCE\xFA\xBE\xBA\x02\x00\x00\x00" >> $tmp
  echo -en $sig | head -c 8 >> $tmp

  md5sum $tmp | head -c 32 > $3
  cat $tmp >> $3
  echo "$3 created"

elif [ $1 == "unpack"  ]; then

#unpack

[ -e $2 ] && echo "firmware folder \"$2\" already exists" && quit
echo "stripping firmware ... "
dd if=$3 skip=2 count=`expr \`stat -c %s $3\` / 16 - 3` ibs=16 obs=4096 of=$tmp &> /dev/null
echo "unpack cramfs image ... "
cramfsck-16k -x $2 $tmp > /dev/null || quit
echo "unpacked to $2"

else

echo usage

fi

quit


[thespecialist]

I just wanted to add, if you, like me, modify a file, flash it, try it, modify, flash, try etc, you might get fed up with editing that md5sum.txt file containing all the md5sums. In that case you can just simply delete the specific entry(ies) from that list ! Not all files are on the list and there's no check (and hence no NEED) at all for them to be on the list.

I think this list is *only* used while flashing anyway (?). However, it brings a (very slight) risk to the table of course and it probably is good practice to not do this when you want to release a certain version to the public.

[b-rad.cc]

no, its checked every boot and things can get very interesting trying to recover a wdtv with failed checksums if the proper files get corrupted.

[thespecialist]

no, its checked every boot and things can get very interesting trying to recover a wdtv with failed checksums if the proper files get corrupted.

Hehe, ok, so deleting from the list probably isnt a very good idea

[g_krassa]

Hey guys

Can this method be used to unpack and repack a wdlxtv firmare. Basically I am interested in repacking brad's 4.2 firmware with a different OSD and to add some custom plugins to the UMSP directory. I was able to unpack repack the the EXT firmware and I am now interested in creating a flash version.

Any guidance would be great to avoid bricking my wdtv.

Thanks